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ABSTRACT 


In  order  to  obtain  an  objective  measure  of  student- 
faculty  contact  during  the  clinical  years  of  the  University 
of  Alberta's  undergraduate  medical  education  program  (third 
and  fourth  year) ,  a  contact-time  survey  was  conducted  during 
the  1975-1976  academic  year. 

Students  were  selected  for  each  week  of  the  study 
period  (fifty-two  weeks)  by  a  probability  sampling  method, 
and  requested  to  fill  out  time  sheets  for  the  one-week 
block.  In  addition,  the  respondents  were  also  asked  to 
estimate,  from  their  point  of  view,  the  percentage  of 
"instructional  component"  for  all  contact  activities. 
Employing  an  appropriate  weighting  mechanism  to  reflect 
unequal  probability  sampling  and  non-responses,  the  results 
were  tabulated  with  respect  to  faculty-hours  per  year  as 
well  as  student-hours  per  week.  Each  tabulation  was  further 
broken  down  by  participating  hospitals,  department  types, 
and  types  of  activity. 

This  study  has  attempted  to  provide  a  base  of  data  with 
which  to  examine  the  relationships  of  student  contact  with 
faculty  during  the  clinical  years  of  undergraduate  medical 
education  at  the  University  of  Alberta. 
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Chapter  1 
INTRODUCTION 


Medical  education  is  heavily  manpower-oriented,  and 
requires  scarce  and  expensive  human  resources.  As  a  result, 
effective  methods  must  be  employed  to  enable  these  resources 
to  be  utilized  to  their  fullest  extent.  Studies  measuring 
manpower  workload  are  often  used  in  industry  for  the  purpose 
of  obtaining  information  pertaining  to  manpower 
requirements.  However,  due  to  the  high  degree  of 
specialization  and  the  professional  nature  required  of 
educators,  and  due  to  the  involved  interactions  between 
educators  and  students,  it  is  generally  known  that  work 
study  methods  have  been  difficult  to  adapt  to  the 
educational  field,  especially  in  professional  programs  such 
as  medical  education. 

1.1  Statement  of  the  Problem 

Recently,  there  has  been  a  "budget  squeeze"  trend 
across  all  publicly-supported  institutions  and  programs, 
including  medical  schools.  This  has  generated  immediate 
pressure  for  administrators  to  review  their  staff  resource 
requirements,  and  either  justify  the  present  level  of 
staffing,  or  examine  alternatives  to  present  resource 
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allocations.  The  present  study  was  initiated  with  the 
objective  of  measuring  the  level  of  teaching  activity 
between  the  medical  undergraduate  students  and  the  faculty. 
More  specifically,  an  attempt  was  made  to  investigate  the 
utilization  level  of  staff  resources,  and  to  measure 
faculty-student  interaction  in  terms  of  contact  time  between 
faculty  and  students,  with  the  intention  that  such  data 
would  assist  administrators  in  their  review  of  staff 
allocations. 

However,  it  must  be  noted  that  this  thesis  examines 
merely  what  was  (and  is)  currently  underway  within  the 
undergraduate  medical  student-internship  program  at  the 
University  of  Alberta,  and  was  not  designed  to  compare  an 
observed  level  of  instructional  activity  with  an  "adequate 
or  desirable"  level  of  instructional  activity.  It  was, 
however,  a  first  step  in  reviewing  the  manpower- 
effectiveness  of  the  Faculty,  and  in  providing  a  data  base 
which,  when  synthesised  with  other  sources  of  information, 
could  be  a  useful  aid  for  manpower  planning,  and  possibly  in 
allocating  funds, 

1,2  Need ,  Demand  and  Utilization 


Ideally,  any  rational  approach  to  resource  allocation 
in  a  service  industry  must  be  based  on  an  assessment  of  the 
needs  of  the  population  served.  However,  in  practice,  one 
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may  face  three  types  of  measures  often  associated  with 
resource  allocation  problems:  that  isr  need,  demand  and 
utilization.  These  concepts  are  certainly  not  identical  nor 
mutually  exclusive,  and  much  theorizing  has  been  attempted 
to  try  and  answer  the  question:  "What  exactly  is  it  that  is 
being  measured?"  Donabedian  (1973,  pp. 58-70)  has  put  forward 
a  theory  describing  the  relationships  between  need,  demand 
and  utilization  in  the  medical  care  setting. 

The  author  will  not,  however,  attempt  to  discuss  the 
sociological  implications  of  need,  demand  and  ultilization 
in  this  thesis,  other  than  to  point  out  that  the  allocation 
and  distribution  of  human  resources  should  ideally  be  based 
upon  measurements  obtained  on  true  system  needs.  However, 
seldom  is  adequate  information  available,  nor  is  it  possible 
to  obtain  at  reasonable  costs. 

In  light  of  this,  and  only  as  a  first  step  in  the 
resource  allocation  process,  the  present  study  is  of  a 
descriptive  nature,  and  attempts  only  to  identify  aspects  of 
utilization.  As  a  result,  the  opinions  of  students  and 
faculty  as  to  the  required  (or  desired)  amount  of 
interactions  are  not  measured  in  this  study.  What  is  being 
measured  are  students'  perceptions  of  their  interaction 
with,  or  utilization  of,  available  instructional  resources. 
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1-3  Background  to  the  Problem 

At  the  University  of  Alberta,  allocations  of  funds  to 
faculties  and  schools  are  usually  associated  with 
utilization  measures  such  as  "student-hours"  and  "contact- 
hours"  (Institutional  Research  and  Planning,  1970).  These 
measures  are  determined  from  the  Registrar's  tabulations  of 
student  enrollment,  which  in  most  cases  consist  of 
allocations  for  formal  lectures,  seminars,  and  laboratory 
sessions-  Following  the  application  of  a  pre-determined 
formula,  a  basis  for  discussion  and  allocation  of  the 
University's  resources  among  faculties  and  schools  is  then 
available  to  the  Board  of  Governors. 

In  applying  measures  such  as  student-hours  and  contact- 
hours  across  the  entire  spectrum  of  faculties  and  schools, 
one  must  assume  that  the  instructional  time  reguirement  for 
student-activity  (ie.  need)  is  approximated  by  the  above 
utilization  measures  which  are  within  certain  margins  of 
tolerable  errors.  However,  within  the  Faculty  of  Medicine, 
thirds  and  fourth-year  undergraduate  medical  students  (Phase 
III)  undergo  much  less  formally-structured  activities. 
Instruction  for  these  students  occurs  almost  exclusively 
within  the  clinical  environment,  mostly  in  hospitals,  as 
part  of  Rotation  and  Elective  programs.  Rotations  and 
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Electives  infer  active  involvement,  on  the  students*  part, 
in  dealing  with  real  patients,  and  in  this  respect,  training 
takes  on  the  form  of  "apprenticeship",  or  more  specifically, 
on-the-job  instruction.  This  deviates  considerably  from 
those  activities  experienced  by  most  other  students  at  the 
University,  and  as  a  result,  it  is  very  difficult,  if  not 
impossible,  to  obtain  universal  need  and/or  demand  measures 
using  the  same  criteria.  In  addition,  where  other  faculties 
and  schools  employ  mainly  full-time  professors,  lecturers 
and  teaching  assistants,  the  undergraduate  student  intern 
program  utilizes  the  services  of  many  part-time  faculty 
members  [physicians,  residents,  and  rotating  interns]  to 
complement  the  services  of  its  full-time  faculty.  As  a 
result,  teaching  activities  are  diluted  by  physicians* 
practices,  hospital  services,  research  and  other  activities. 

1.4  The  Pilot  Study 

In  the  Spring  of  1975,  a  feasibility  study  of  Phase  III 
faculty-student  contact  time  was  designed  [Grady,  W., 
Makowsky,  M. ,  Mitchell,  I.,  Teixeira,  C.,  1975].  This  pilot 
study  was  conducted  to  test  the  applicability  of  two 
alternate  research  instruments.  Based  on  the  pilot  study,  it 
was  recommended  by  its  authors  that  such  a  study  be  carried 
out.  Furthermore,  a  positive  reaction  by  the  Phase  III 
students  of  the  Faculty  of  Medicine  was  indicated  by  the 
results  of  an  opinion  survey  of  the  respondents.  The 
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students'  opinions  indicated  that  the  research  instrument's 
instructions  were  clear  and  concise,  that  the  instrument 
used  to  collect  the  data  was  relatively  easy  to  fill  out, 
that  it  took  less  than  five  minutes  per  day  to  do  so,  and 
that  the  students  did  not  object  to  completing  a  weekly 
time-sheet  3-4  times  per  year. 

1.5  Objectives  of  the  Study 

Based  on  the  findings  of  the  pilot  study,  the  present 
investigation  was  undertaken  with  the  objective  of 
estimating  Phase  III  faculty-student  contact-time  as 
reported  by  the  students.  More  specifically,  the  author 
attempted  to  derive  a  utilization  measure  of  faculty 
resources,  namely: 


(1)  To  estimate  the  actual  amount  of  faculty- 

student  contact-time  over  both  Rotation 
and  Elective  activities, 

(2)  To  compare  contact-time  amongst  participating 

teaching  hospitals  and  departments, 

(3)  To  estimate  and  compare  the  amount  of  direct 

instruction  received  by  the  students  in 
the  various  hospital  departments,  as 
perceived  by  the  students  themselves, 
and 

(4)  To  evaluate  the  methodology  used  to  extract 

the  above  information  from  the  students. 


 .  . . 
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1.6  Scope  of  the  Study 

The  study  focused  on  the  direct  contact  time  between 
Phase  III  medical  students  and  their  faculty  counterparts. 
The  time  frame  of  the  data  acquisition  phase  was  from 
September  4f  1975  to  September  2,  1976,  giving  a  total  of  52 
calendar- weeks. 

The  study  was  limited  to  those  students  participating 
in  the  Rotation  and  Elective  programs,  and  subsequently 
reflected  upon  only  those  faculty  members  and  other  resource 
personnel  directly  involved  with  the  programs. 

The  following  Alberta  hospitals  were  included  for 
regular  Rotation: 

(1)  University  of  Alberta  Hospital 

(2)  Royal  Alexandra  Hospital 

(3)  Edmonton  General  Hospital 

(4)  Misericordia  Hospital 

(5)  Alberta  Hospital  (Oliver) 

(6)  Charles  Camsell  Hospital 

Furthermore,  departments  within  hospitals  were  limited  to 
Medicine,  Surgery,  Obstetrics  and  Gynecology,  Paediatrics, 
and  Psychiatry.  The  activities  of  Medicine  were  divided  into 
two  sequential  groups,  owing  to  the  large  curricular  load, 
and  were  denoted  by  Medicine  I  and  Medicine  II.  In  addition. 
Elective  activities  were  measured  as  if  they  were  grouped  as 
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a  unique  department. 

Faculty-student  contact  time  was  restricted  to  direct 
instructional  activities.  This  implies  that  casual,  non- 
instructional  contact  by  the  students  with  faculty  members 
was  not  included  in  the  study. 

1.7  Methodological  Limitations 

The  following  were  limitations  encountered  by  the 
author  in  carrying  out  the  study: 

(1)  The  study  methodology  was  somewhat  constrained 
because  there  was  little  prior  information  available  to 
construct  an  "optimum"  sampling  design. 

(2)  Conceptually,  it  is  more  desirable  to  obtain 
information  pertaining  to  manpower  requirements  by  way  of  a 
normative,  rather  than  a  descriptive  study.  However,  due  to 
the  lack  of  desirable  standards,  and  due  to  measurement 
difficulties,  the  study  attempted  to  measure  what  was 
happening  at  the  time,  and  was  not  designed  to  compare  the 
results  with  what  was  supposed  to  be  happening,  nor  to  make 
value  judgements.  Furthermore,  items  relating  to  the  quality 
of  instruction  were  intentionally  omitted  from  the 
instrument.  Nevertheless,  an  attempt  was  made  to  quantify 
the  instructional  component,  but  admittedly,  the  method  is 
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limited  and  subject  to  criticism. 

1.8  Definitions  Pertaining  to  the  Study 

The  following  is  a  list  of  definitions  of  terms  used 
throughout  this  thesis: 


1.  Phase  III  Student: 

A  student  registered  in  the  Faculty  of 
Medicine  at  the  University  of  Alberta,  who  is 
undergoing  his/her  third  or  fourth  year  of 
undergraduate  study. 

2.  Student  Contact  Time: 

The  time  spent  in  sessions  during  which  Phase 
III  students  are  observing  or  receiving 
instruction  under  direct  supervision  of  one  or 
more  preceptors  and/or  other  medical  staff, 
including  residents  and  interns.  Direct 
supervision  is  defined  as  contact  during  which 
both  the  faculty  member  and  the  student  are  within 
eyesight  of  one  another. 

3.  Student  Week: 

A  period  of  seven  consecutive  days,  starting 
Thursdays  at  9:00  A.M.,  spent  by  a  Phase  III 
student  as  a  part  of  his/her  formal  rotation  or 
elective  schedule.  Thursdays  were  chosen  as  the 
commencement  of  a  rotation  or  elective  to  coincide 
with  the  schedule  assigned  by  the  Faculty  of 
Medicine. 

4.  Per  Cent  Instructional  Component: 

The  proportion  of  activity  relevant  to 
instruction,  estimated  by  Phase  III  students,  for 
any  contact  activity.  This  is  a  quantitative 
measure  only,  and  is  not  intended  to  reflect  any 
qualitative  aspect. 

5.  Contact  Activities: 

Regular,  direct  activities  through  which  a 
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Phase  III  student  experiences  potential  contact 
with  preceptors  and  other  staff  involved  in  the 
Phase  III  program.  Examples  of  Contact  Activities 
are:  ward  rounds,  admissions  review,  clinical 
rounds,  operating  room  sessions,  clinics, 
seminars,  lectures,  and  night  calls. 

6.  Rotation  and  Elective  Week: 

One  out  of  several  weeks  required  of 
(rotation)  or  selected  by  (elective)  a  student  to 
formally  spend  in  a  particular  hospital  and 
department  as  a  student  intern. 

7.  Calendar  Week: 

Concurrent  with  a  Rotation  or  Elective  week, 
but  with  respect  to  the  study  time  frame.  For 
example.  Calendar  Week  #1  was  from  September  4, 
1975  to  September  11,  1975. 

8.  Cohort: 

Students  receiving  Phase  III  instruction  as  a 
group.  In  this  study,  groups  were  surveyed  during 
a  4-,  6-,  or  8-week  period  [depending  upon  the 
Rotation  or  Elective  schedule]. 


1.9  Summary  of  the  Thesis  Format 

A  review  of  pertinent  literature  will  be  found  in 
Chapter  2,  followed  by  a  presentation  of  the  study 
methodology  in  Chapter  3.  Results  and  interpretations  are 
discussed  in  Chapter  4,  and  the  summaries,  conclusions,  and 
recommendations  are  presented  in  Chapter  5. 
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Chapter  2 

THE  APPLICATION  OF  OPERATIONS  RESEARCH 
IN  MEDICAL  EDUCATION: 

A  LITERATURE  REVIEW 


The  concept  of  operations  research  (often  called 
systems  analysis  or  management  science) 1  has  been  a  part  of 
modern  management  for  most  of  the  past  decade  (Wagnerr 
1969).  The  literature  is  saturated  with  examples  of  how 
operations  research  has  been  used  to  increase  efficiency, 
reduce  operating  costs,  and  optimize  resource  allocations  in 
the  private,  and  some  public  sectors  of  the  economy. 
Governments  and  other  public  bodies  have  found  operations 
research  useful  in  enabling  large  departments  to  operate  in 
the  face  of  escalating  costs  and  constrained  budgets.  As  the 
level  of  technology  increases  in  all  aspects  of  public  and 
private  enterprise,  the  need  for  rational  management 
decision-making  becomes  essential  in  maintaining  efficient 
provision  of  goods  and  services  and  control  of  costs.  To 


1  Strictly  speaking,  these  refer  to  different 
methodologies,  but  substantial  components  overlap  and  are 
quite  often  used  interchangably .  For  example,  Connors ( 1 975) 
"...doubts  whether  much  concensus  could  be  achieved  on 
definitions  of  such  crucial  terms  as  operations  research, 
systems  analysis,  systems  engineering,  management 
engineering,  industrial  engineering,  system,  output,  input, 
process,  and  so  on  [p.22]." 
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this  end,  operations  research  has  become  an  established  and 
widely-used  aid. 

Although  fairly  new  to  the  management  of  health  care 
services,  operations  research  has  been  utilized  frequently 
in  the  past  decade.  The  purpose  of  this  chapter  is  to  review 
the  pertinent  literature  of  operations  research,  time-study, 
and  manpower-planning,  as  well  as  to  illustrate  their 
applicability  to  research  in  medical  education. 

2. 1  Operations  Research 

Due  to  the  potentially  widespread  applicability  of 
operations  research,  there  exists  no  universally  agreed-upon 
definition  of  this  relatively  new  discipline.  Attempts  have 
led  to  generalities  such  as  "the  scientific  approach  to 
decision-making  that  involves  the  operations  of 
organizational  systems"  (Hillier  and  Lieberman,  1970,  p.5). 
Koza  (1973)  suggests  "..  a  very  general  definition  of 
operations  research  may  be  stated  as  ...the  application  of 
scientific  methods,  techniques,  and  tools  to  problems 
involving  the  operations  of  systems  solutions  to  the 
problems  [p.  1]."  And  Wagner  (1969),  "for  convenience", 
defines  operations  research  as  "a  scientific  approach  to 
problem-solving  for  executive  management  ( p.  4]." 
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Although  the  definitions  may  vary  substantially,  the 
following  keywords  seem  to  be  quite  common:  science, 
mathematical  model,  decisions,  systems,  and  management.1 

Hillier  and  Lieberman  (1970,  p.5)  maintain  that  the 
best  way  of  examining  the  nature  of  operations  research  is 
to  outline  its  characteristics.  First  of  all,  the  breadth  of 
applications  is  unusually  wide,  including  business, 
industry,  the  military,  civil  government,  agencies,  and 
hospitals.  Secondly,  the  approach  of  operations  research  is 
that  of  the  scientific  method;  specifically,  the  formulation 
of  the  problem  and  the  construction  of  a  scientific 
(typically  mathematical)  model  that  attempts  to  abstract  the 
essence  of  the  real  problem.  Experimentation  is  the  third 
phase:  testing  the  applicability  of  the  model  to  the 
problematic  reality  (often  by  simulation  methods) ,  followed 
by  the  formulation  of  conclusions  by  the  decision-makers 
when  they  are  needed.  Hillier  and  Lieberman  (1970)  summarize 
by  stating  that  "operations  research  involves  creative 
scientific  research  into  the  fundamental  properties  of 
operations...  [and  concerns]  the  practical  management  of  the 
organization  [p.  5]." 

Wagner  (1969,  p.  5)  refuses  to  accept  operations 


1  The  author  does  not  attempt  to  discuss  systems  theory 
or  decision  theory  in  this  text.  Interested  readers  are 
invited  to  consult  Churchman,  C.W.,  The  Systems  Approach, 

New  York:  Delta,  1968. 
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research  as  an  approach  to  problem-solving  unless  it 
contains  certain  qualities:  (1)  a  primary  focus  on  decision¬ 
making,  (2)  an  appraisal  resting  on  economic  effectiveness 
criteria  [i.e.  costs,  revenues,  cash  flow,  rate  of  return, 
etc- ],  (3)  reliance  on  a  formal  mathematical  model,  and  (4) 

dependence  on  electronic  computers  [to  facilitate  complex 
computations  and  large  volumes  of  data  ]. 

A  study  in  1965  (Schumacher  and  Smith,  1965)  provided 
an  overall  picture  of  operations  research  activities  in 
private  industry  at  that  time-  The  primary  activities 
included  forecasting,  production  scheduling,  inventory 
control,  quality  control,  transportation,  advertising  and 
sales  research,  maintenance  and  repairs,  accounting 
procedures,  plant  location,  equipment  replacement, 
packaging,  and  capital  budgeting. 

2- 2  Time  Study 

One  specific  application  of  operations  research  is 
time-study-  Niebel  (1973)  defines  time-study  as  "the 
technique  of  establishing  an  allowed  time  standard  to 
perform  a  given  task,  based  upon  measurement  of  work  content 
of  the  prescribed  method  [p-  6]."  Techniques  useful  to  time- 
study  include  stopwatch  study,  standard  data,  fundamental 
motion  data,  work  sampling,  and  estimates  based  on 
historical  data  (Niebel,  1973,  p. 6) - 
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Modern  time-study  began  in  1881,  as  a  result  of  the 
efforts  of  Frederick  W.  Taylor,  considered  to  be  the  father 
of  the  scientific  management  movement.  By  1917,  many 
industrial  plants  had  adopted  Taylor's  "shop  management" 
techniques  with  considerable  success  in  increasing 
efficiency  (Thompson,  1917). 

Men  such  as  Frank  Gilbreth,  Carl  Barth,  Henry  Gantt, 
and  Morris  Cooke  contributed  to  and  developed  Taylor's 
original  ideas,  and  by  World  War  II,  time-study  in  its 
contemporary  form  was  a  well-established  analytical 
procedure  in  the  United  States.  Since  then,  many 
associations  have  been  formed  for  the  express  purpose  of 
"bringing  the  science  of  time-study...  up  to  present-day 
standards  [Niebel,  1973,  p. 15]". 

However,  the  current  trend  is  tending  to  de-emphasize 
scientific  management  techniques,  and  is  moving  more  towards 
a  management  science  approach.  For  example,  authors  such  as 
Churchman  (1968)  have  recently  questioned  the  benefits  of 
scientific  management  techniques,  on  the  grounds  that  the 
"human  aspect"  has  been  neglected,  and  the  system's  goal  has 
not  been  "optimized"  [i.e.  efficiency  but  not 
effectiveness].  It  is  generally  agreed  upon  that  time  study 
is  difficult  to  apply  to  industries  of  a  service  nature, 
such  as  in  medical  education,  due  to  the  manpower- 
intensi veness  and  the  dominance  of  human  relations. 
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2. 3  Operations  Research  and  Health  Care 

The  health  care  industry  has  not  escaped  the  rapid 

evolution  of  technology.  Wanielista  (1970)  is  quite  correct 

in  observing  that  "with  the  increased  use  of  scientific 

techniques  in  management,  the  health  care  administrator  is 

finding  a  new  environment  in  which  to  conduct  his  planning 

and  operations  [ p.  i]".  He  goes  on  to  say  that  operations 

research  has  gained  the  acceptance  of  the  profession  in 

health  care  planning,  medicare  administration,  and  medical 

research  (on  the  macro- level) ,  as  well  as  in  work 

simplification  and  manpower- planning  (on  the  micro-level). 

Koza  (1973)  points  out  that: 

...today's  health  administrators  must  learn  to  use 
the  quantitative  information  that  can  be  provided 
by  mathematical  and  operations  research  techniques 
as  an  aid  in  the  decision-making  process  [ p.  1 ]. 

The  use  of  management  science  approaches  in  health 
manpower  studies  was  frequent  during  the  1950*s  and  1960's. 
During  this  time  period,  the  bulk  of  operations  research 
projects  involved  time  studies  of  nursing  activities  and 
detailed  analyses  of  the  functions  of  professional  nurses 
(Levine  and  Kahn,  1975,  p.338) . 

Young  (1975,  p.xi)  has  stated  that  since  the  early 
1960's,  the  amount  of  research  effort  in  health  has 
increased  exponentially.  However,  beyond  a  few  initial 
analytical  breakthroughs,  the  potential  of  operations 
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research  in  health  care  has  perhaps  not  been  as  fully 
realized  as  might  be  expected. 

Management  scientists  have  begun  to  exercise  caution  in 
the  "blind"  use  of  operations  research  techniques  in  the 
health  care  field.  As  in  any  social  science  research,  they 
must  deal  with  complex  human  behavior  which  present 
challenging  measurement  problems.  Authors  agree  that 
operations  research  is  but  a  guideline,  a  mere  aid  to 
management  in  decision-making,  and  should  never  be  relied 
upon  entirely.  Stimson  and  Stimson  (1972)  argue  that 
operations  research  has  been  less  than  enthusiastically 
received  amongst  health  care  [especially  hospital] 
administrators.  The  major  difficulties  in  the  implementation 
of  operations  research  studies  in  hospitals  have  been 
identified  as  the  following: 


1.  Omission  of  some  problems  deemed  important 
by  administrators,  such  as  quality  of  care, 
continuity  of  care,  and  hospital-community 
relationships. 

2.  Tendency  to  view  the  hospital  as  a 
separable  system,  whereby  interrelations  between 
parts  of  the  organization  are  neglected. 

3.  Tendency  to  view  the  hospital  solely  as  a 
mechanistic  system,  failing  to  identify  it  as  a 
sensitive  social  system  as  well. 

4.  Lack  of  attention  to  model-building  and  to 
the  way  data  for  models  are  generated. 

5.  Omission  of  variables  that  are  difficult 
to  quantify,  such  as  the  quality  of  patient  care. 
[Stimson  and  Stimson,  1972,  pp. 41-64] 
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Nevertheless,  Stimson  and  Stimson  are  not  entirely 
opposed  to  the  use  of  operations  research  in  the  health  care 
sector:  ’’The  challenge  for  operations  researchers  is  to 
develop  theory  and  methods  for  studying  in  a  holistic  manner 
the  complex  system  that  is  the  hospital  [Stimson  and 
Stimson,  1972,  p.83]." 

2.4  The  Need  for  Research  in  Medical  Education 

The  major  human  resource  in  the  health  industry  is,  of 
course,  the  physician.  It  stands  to  reason,  therefore,  that 
improvements  made  to  programs  of  physician  training  will 
ultimately  reflect  upon  improved  health  care  in  the  general 
sense.  In  a  report  to  the  Ontario  Council  of  Health  on 
Health  Manpower  (1969,  p.83),  it  was  revealed  that,  with  the 
continuing  trend  of  admitting  more  Canadians  into  medical 
schools  each  year,  and  despite  the  reduction  of  physician 
immigrants  to  Canada  (49%  of  newly-registered  physicians 
were  immigrants  in  1968),  by  1986  Ontario  would  boast  one 
physician  for  every  900  Ontario  residents.  The  report 
urgently  recommended  a  program  of  study  and  planning  for 
health  education. 

The  steady  increase  of  costs  to  the  taxpayer  required 
to  educate  physicians  indicates  a  strong  need  for  intensive 
research.  In  1968,  the  average  expenditure  per  medical 
school  in  the  United  States  was  $5.5  million,  as  compared 
with  $2.6  million  in  1961  (Entwisle,  1971).  Should  present 
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trends  continue,  it  is  forecasted  that,  by  1980,  the  average 
annual  expenditure  per  medical  student  will  approach  $44,000 
(Entwisle,  1971) . 

The  average  annual  cost  of  educating  a  Canadian  medical 
student  in  1967  was  $4,925  for  undergraduates,  and  $8,827 
for  graduate  students  (Association  of  Universities  and 
Colleges  of  Canada,  1970,  p.  161).  At  the  University  of 
Alberta's  Faculty  of  Medicine,  the  1970  cost  per 
undergraduate  student  in  each  of  the  four  years  of  study  was 
$5,033.,  $3,292.,  $5,958.,  and  $6,084.,  respectively 
(Institutional  Research  and  Planning,  1971,  p. 163). 

It  should  be  noted,  however,  that  the  U.S.  cost  figures 
are  not  directly  comparable  with  the  Canadian  figures.  This 
is  partly  due  to  the  different  financing  mechanisms 
governing  health  care  delivery  in  the  two  countries. 

Although  all  the  cost  information  is  well- documented ,  the 
assumptions  made  in  the  calculation  of  the  values  are  not. 
Therefore,  the  reader  must  consider  the  figures  separately, 
and  not  attempt  to  make  direct  comparisons. 

The  interrelation  of  systems  can  be  illustrated  in  the 
cost  of  medical  education.  As  Foster  (1972)  points  out, 
"medical  education  will  continue  to  be  heavily  patient  and 
hospital  centred,  and  as  long  as  this  is  true  the  hospital 
will  continue  to  be  a  teaching  as  well  as  a  service 
institution  [p.  9]."  What  is  implied  is  that  the  hospital 
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must  share  in  the  burden  of  financing  medical  education,  and 
that  the  patient  is  an  indirect  contributor-  In  Canada,  the 
per  patient  operating  costs  of  a  teaching  hospital  are  twice 
those  of  a  non-teaching  hospital  (MacFarlane,  1964,  p-122), 
and  the  situation  seems  to  be  quite  similar  in  Britain 
(Hill,  1968,  p. 117).  Although  case  mix  differential  may 
account  for  a  substantial  portion  of  this  difference,  it  can 
be  postulated  that  the  educational  component  is  indeed  a 
factor  responsible  for  increased  costs. 

Vodev  (1957,  p.  107)  advocates  that  every  effort  should 
be  made  to  transfer  education  costs  from  the  patient  to 
broader  community  support.  This  is  already  the  case  in 
Canada,  due  to  universal  medicare  and  government  support  of 
hospital  costs. 

2-5  Evaluation  Studies  in  Medical  Education 

The  use  of  on-going  evaluation  techniques  has  been 
deemed  an  essential  control  mechanism  in  adapting  to  a 
changing  technological  and  social  environment-  On  a  micro¬ 
level,  evaluation  serves  as  a  decision-making  tool:  ’’-.-[an] 
instrument  for  making  wiser  choices  from  a  set  or  sets  of 
alternatives  that  are  outgrowths  from  what  has  occurred  and 
what  is  to  follow  ...a  commitment  to  better  quality  control 
of  the  end  product  [Blain,  1972,  p.16]." 


Education,  in  general,  has  historically  seen  much 
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attention  devoted  to  the  development  and  use  of  evaluation 
techniques.  For  example,  examinations  evaluate  students' 
achievement;  "course  evaluations"  assess  teachers' 
performance;  and  accreditation  teams  evaluate  academic 
standards  of  teaching  programs.  Although  some  significant 
progress  has  been  made  in  educational  and  psychological 
measurement  [especially  in  the  area  of  aptitude  tests],  the 
evaluation  has  not  been  any  more  successful  in  education 
than  in  other  social  programs. 

Medical  faculties  at  universities  have  also  been 
subject  to  much  evaluative  research  covering  a  broad 
spectrum.  For  this  purpose,  as  well  as  to  satisfy  other 
management  needs,  a  medical  school  must  supply  information 
on  a  broad  range.  Kutina  and  Lee  (1973)  have  defined  a  basic 
set  of  data-base  requirements  for  a  medical  school 
information  system.  Aside  from  financial  and  budgetary  data, 
the  authors  included  data  of  personnel  effort,  curricula, 
facilities,  and  student  activity.  This  system  was 
implemented  at  the  School  of  Medicine,  Case  Western  Reserve 
University,  with  the  purpose  of  strengthening  the  ability  of 
the  administration  to  plan  and  manage  the  affairs  of  the 
school.  The  project  determined  that  the  information  gathered 
was  most  adequate  "for  external  reporting,  day-to-day 
internal  operating,  and  fiscal  auditing  purposes  [Kutina  and 
Lee,  1973,  p.803]«"  Unfortunately,  no  comments  on  the 
implications  for  planning  and  allocating  resources  were 
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reported. 

Blain  (1972)  and  members  of  the  Educational  Resources 
Group,  School  of  Medicine,  University  of  Missouri  at 
Columbia,  conducted  studies  on  medical  educators,  health 
care  education  professionals,  and  medical  and  health  care 
students.  One  of  the  specific  projects  included  the  mailing 
of  questionnaires  to  students,  and  this  proved  useful  in 
determining  feedback  regarding  the  quality  of  courses  and 
instruction. 

Among  the  many  documented  resource  planning  information 
systems  currently  in  use  in  educational  institutions,  there 
has  recently  appeared  interest  in  the  development  of  health 
education  planning  models.  Systems  such  as  CAMPUS, 
HELP/PLANTRAN,  RRPM ,  and  SEARCH  have  been  developed  to  help 
"predict  cost  and  resource  requirements  by  analyzing  the 
forecast  enrollment  by  discipline  and  assessing  the  impact 
of  alternative  planning  policies  [Systems  Dimensions 
Limited,  1974,  p.70]H.  However,  because  the  emergence  of 
systems  planning  models  has  occured  only  in  relatively 
recent  years,  there  still  exists  some  concern  over  their 
practical  application. 

The  Faculty  of  Medicine  at  the  University  of  Alberta 
has,  in  the  past,  attempted  to  estimate  faculty  teaching 
workload  using  survey  techniques.  Two  studies  were 
conducted,  the  first  occuring  in  April  of  1971,  and  the 
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second  during  the  1973-1974  academic  year.  Both  studies 
resulted  in  unsatisfactory  estimates  of  staff  teaching 
workloads,  due  mainly  to  lack  of  committment  by  the  faculty 
respondents  (poor  response  rate) ,  and  apparent  overestimates 
provided  by  the  responding  faculty  members.1 

It  should  be  noted  the  above  studies,  as  well  as  the 
majority  of  those  found  in  the  literature,  measured 
utilization  through  response  from  faculty- members.  Hilles 
(1973)  attributed  the  shortcomings  of  faculty  effort¬ 
reporting  to: 


(D  The  stigma  which  effort  reporting  carries  [a 
time-clock-punching  connotation ]. 

(2)  Lack  of  objectivity  in  separating  the  teaching 

component  from  the  service  component, 

(3)  Bias  is  inherent  in  the  effort  technique  which 

varies  with  the  source  of  the  faculty*s 
salary.  If  a  faculty  is  supported  mainly 
from  research  funds,  one  would  expect 
bias  to  exhibit  a  high  proportion  of 
time  to  research. 

(4)  The  subjectivity  of  the  effort  reporting 

technique  in  that  faculty  must  provide 
estimates  of  effort  [p.  806]. 


Stoddart  (1973)  identified  a  major  weakness  in  faculty 
questionnaire  reporting,  that  being  the  tendency  of 


1  This  information  was  obtained  in  a  series  of 
conversations  between  the  author  and  Mr.  H.  Fearon,  Office 
of  the  Dean  of  Medicine,  Faculty  of  Medicine,  University  of 
Alberta. 
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individuals  to  reflect  upon  their  most  recent  pattern  of 
activity.  The  characteristic  low  response  rate  from  faculty 
members  precludes  the  use  of  on-going  measurement 
techniques. 

Most  researchers  agree  that,  in  order  to  obtain 
accurate  information,  faculty-respondents  must  be  able  to 
separate  teaching  activities  from  all  other  activities,  and 
in  this  respect,  most  studies  have  proved  unsatisfactory 
(Smythe,  1970;  Hilles,  1973;  Stoddart,  1973). 

2. 6  Medical  Clerkships  and  Faculty-Student  Contact 

Until  recently,  senior  medical  students  followed  a 

fairly  structured  curriculum  format.  J.A.  MacFarlane,  then 

the  director  of  the  Royal  Commission  on  Health  Services, 

identified  the  shortcomings  of  this  format  type,  and 

proposed  a  somewhat  radical  change: 

In  most  Canadian  schools,  the  final  year  students 
are  given  only  limited  responsibility...  they  are 
being  taught  most  of  the  day,  rather  than  being 
left  on  their  own  with  the  faculties  for 
supervised  learning  on  the  wards  and  in  the 
libraries  and  laboratories  [MacFarlane,  1961, 
p .  270  ]. 

Some  years  later,  when  the  Commission  published  its 
report  on  Medical  Education  in  Canada,  MacFarlane  noted  that 
medical  schools  in  most  countries  other  than  Canada  utilized 
what  was  commonly  called  the  "Block  System".  Under  this 
system,  a  student  is  assigned  to  a  hospital  service,  and 
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actually  becomes  a  part  of  that  unit  with  definite 

responsibilities.  All  of  the  student's  time  is  devoted,  for 

a  designated  period,  to  studies  in  that  particular  area  and 

to  the  activities  of  the  unit  (MacFarlane,  1964,  p.80).  The 

Commission  made  its  recommendation: 

We  think  that  the  student,  during  his  final  year, 
is  sufficiently  mature  to  do  much  of  the  work  that 
is  now  allocated  to  the  so-called  'rotating 
intern'  in  the  teaching  hospital...  the  student 
could  be  assigned  to  this  duty  as  a  clinical  clerk 
[MacFarlane,  1964,  p.81]. 

The  "Block  System"  has  since  been  adopted  in  Canadian 
medical  schools,  including  that  of  the  University  of 
Alberta,  and  has  been  operating  successfully  for  several 
years.  However,  because  the  clinical  clerks  are  only 
partially  supervised,  it  has  been  difficult  for  the  medical 
school  administrators  to  evaluate  the  program  in  terms  of 
curriculum,  faculty  effort,  and  student  performance.  As 
Murphee  (1972)  states: 

Maintaining  regular  faculty  contact  with  the 
students  during  the  clerkship  requires  both  travel 
time  and  funds.  Implementing  a  successful 
evaluation  of  such  a  dispersed  program  demands 
minute  planning  and  preparation  of  paper  work 
[ p. 930  ]. 

Evaluation  of  the  so-called  "clinical  experience"  of 
the  clerkship  has  been  focused  upon  "...the  qualifications, 
motivation  and  attitudes  of  the  clinical  supervisor. . .  [and] 
the  ratio  of  clinical  supervisors  to  students"  (Enright, 
1972,  p-159).  Two  studies,  presented  in  the  following 
sections,  have  recently  been  attempted  in  Canada  with  the 


e4T 


. 


■ 


26 


purpose  of  evaluating  faculty- student  contact  in  clerkship 
programs.  One  study  asked  faculty  members  to  report  on 
contact  activities ,  and  the  other  evaluated  responses  from 
the  students  regarding  contact  activities.  A  third  study  is 
also  presented  illustrating  a  complex  sampling  methodology 
used  to  investigate  physician- patient  contact  time. 

2.7  The  FPTU  Study* 

Stoddart  (1973)  and  his  associates  designed  an  activity 
reporting  methodology  to  be  employed  "as  part  of  an  overall 
methodology  of  a  cost  analysis  of  the  Family  Practice 
Teaching  Unit  (FPTU)  of  the  University  of  British  Columbia 
Department  of  Health  Care  and  Epidemiology  [ p.  820]."  The 
objectives  of  the  unit,  as  defined,  were: 

(1)  To  provide  educational  experience  for  health 

science  undergraduates  as  well  as  first- 
and  second-year  residents  in  family 
medicine. 

(2)  To  supply  primary  health  care  to  families  in 

the  community  [  p.  820  ]. 

The  overall  objective  of  the  study  was  to  determine  the 
costs  of  the  education  and  patient  care  functions  of  the 
unit. 


i  A  discussion  cf  the  results  of  this  study  can  be 
found  by  consulting  Stoddart,  G.L. ,  "Effort  Reporting  and 
Cost  Analysis  of  Medical  Education",  Journal  of  Medical 
Education,  1973,  48(9),  814-23. 
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Activity  reports  were  completed  by  all  non-student  FPTU 
personnel  (sample  size  was  31)  on  the  basis  of  the  preceding 
year's  work.  Respondents  were  asked  to  estimate  the 
percentage  of  time  spent  in  the  following  procedures: 

(1)  Patient  contact  without  students. 

(2)  Patient  contact  with  students  present. 

(3)  Direct  contact  with  students  (no  patients 

present) . 

(4)  Conferences 

(5)  Rounds 

(6)  Administration 

(7)  Clerical  functions 

(8)  Self-Education 

In  addition,  the  respondents  classified  the  activities  into 
three  categories:  patient  care,  education,  "other",  or  some 
combination  of  each  (reported  by  percentage) . 

The  activity  reports  were  reviewed  with  each  respondent 
by  a  "knowledgeable  observer",  mainly  to  ensure  100% 
response  taking  into  account  the  small  sample  size.  Both 
Stoddart  and  Hilles  (1973)  identified  the  tendency  of 
respondents  to  reflect  only  upon  their  most  recent  pattern 
of  activity,  despite  the  fact  that  the  information  collected 
was  to  represent  the  activities  of  the  entire  preceding 
year. 


The  data  collected  from  the  activity  reports  was  used 
to  attribute  direct  salary  expense  for  each  individual  to 
the  education  and  patient  care  functions,  as  well  as  to 
establish  allocation  criteria  for  supply  expenses  and 
general  overhead. 
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Amongst  the  sources  of  error  in  the  study  was  the 
confusion  of  the  respondents  over  "conferences"  and 
"rounds".  Stoddart  (1973)  recommended  the  amalgamation  of 
the  two  categories  in  future  studies  of  a  similar  nature.  In 
addition,  it  was  found  that  respondents  "found  it  more 
difficult  to  estimate  the  percentage  of  their  time  spent  on 
procedures  than  to  divide  the  percentages  up  among  the  three 
activity  categories  [  p.  822  ]". 

Stoddart  suggested  that  the  study  might  be  repeated, 
including  the  health  science  students  in  the  reporting 
process.  The  rationale  behind  this  was  to  provide  estimates 
for  opportunity  costs  (the  value  of  the  students'  time)  and 
replacement  costs  (the  cost  of  hiring  persons  to  perform  the 
work  currently  done  by  students) . 

The  investigator  concluded  his  report  with  the 
assurance  that,  although  his  study  only  provided  subjective 
estimates,  his  activity  reporting  technique  "produces  more 
accurate  estimates  of  the  relative  costs  of  patient  care  and 
education  in  clinical  teaching  settings  than  would  otherwise 
be  obtained  [  p*  823]." 


. 
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2.8  The  ACMC  Study 1 

Entitled  "Evaluation  of  clerkship  in  Canadian  medical 
schools",  a  research  project  was  initiated  by  the 
Association  of  Canadian  Medical  Colleges  (ACMC)  during  the 
1975/76  academic  year.  The  study  evolved  as  a  result  of 
conflicting  opinions  as  to  the  value  of  the  clinical 
clerkship  (or  undergraduate  clinical  training)  vis  a  vis  the 
rotating  internship  (post-graduate) .  Many  believed  the 
educational  value  of  the  clerkship  is  superior  to  that  of 
the  internship,  others  believed  it  to  be  equivalent,  and 
still  others,  inferior. 

In  addition,  discussions  were  underway  regarding 
licensing  requirements,  especially  those  in  family  practice. 
Some  considered  one,  two,  or  even  three  years  of  residency 
necessary  for  licensing,  however  many  students  wrote  [and 
passed]  their  licensure  examinations  even  before  they  began 
their  post-graduate  training. 

As  a  result,  the  ACMC  initiated  the  research  project 
with  the  objective  of  evaluating  the  clerkship  programs 
across  Canada,  and  their  implication  for  post-graduate 
training.  The  project  format  took  on  two  phases:  (1)  daily 


i  During  the  time  the  ACMC  study  was  in  progress,  all 
information  was  obtained  through  correspondance  with  Dr.  Guy 
Lamarche,  Association  of  Canadian  Medical  Colleges,  151 
Slater  St.,  Ottawa,  Ontario  KIP  5H3. 
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activities  of  the  clerk,  and  (2)  on  site  visits  of  all  [16] 
schools. 

Of  the  1768  clinical  clerks  registered  in  Canada  in 
1975/76,  approximately  530  were  selected  as  respondents. 
This  was  accomplished  by  asking  each  medical  school  to 
select,  without  guidance,  a  random  sample  of  30%  of  their 
clerks.  All  selected  clerks  were  required  to  complete  a 
booklet  of  minute  details,  for  a  period  of  three  weeks. 


Each  respondent  was  asked  to  report  the  design  of 
his/her  clerkship  program,  including  the  number  of  weeks 
[and  the  location]  of  each  required  Rotation  and  each 
selected  Elective.  In  addition,  for  every  half-hour  period 
during  the  three  weeks,  the  respondent  was  requested  to 
report  details  on  activities,  setting,  format,  and 
evaluation.  The  suggested  activities  included: 

(1)  history  taking 

(2)  physical  examination 

(3)  operating  room  work 

(4)  emergency  room  work 

(5)  laboratory  work 

(6)  out-patient  work 

(7)  grand  round 

(8)  lecture,  seminar,  or  tutorial 

(9)  breaks  (coffee,  lunch,  dinner,  holiday, 

travel) 


A  setting  was  defined  as  follows: 

(1)  wards 

(2)  operating  room 

(3)  emergency  room 

(4)  doctor's  office 

(5)  lecture  theatre 

(6)  automobile 
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Format  was  meant  to  include: 

(1)  alone 

(2)  with  patient 

(3)  with  intern  (or  teacher,  resident,  nurse-.) 

(4)  small  or  large  groups 


For  the  section  on  evaluation,  the  respondent  was 
requested  to  rate  each  activity  on  a  scale  from  0  to  4  in 
terms  of  the  educational  value  of  that  activity.  A  score  of 
0  indicated  no  value,  and  a  score  of  4  implied  excellent 
educational  value. 

The  ACMC  study  was  coordinated  at  each  school  under  the 
auspices  of  the  clerkship  program  director.  The  random 
selection  of  the  30%  sample  became  his  responsibility ,  and 
the  selection  method  inevitably  differed  from  school  to 
school.  In  addition,  all  respondents  were  asked  to  report 
during  the  same  three-week  period.  This  procedure  failed  to 
take  a  time  factor  into  account,  and  merely  provided  a 
crossectional  account  of  clerkship  activities  across  the 
country  over  the  designated  three  weeks. 

Phase  II,  or  the  on  site  visit  stage,  included 
interviews  with: 

(1)  clerks 

(2)  clerkship  program  directors 

(3)  faculty  members 

(4)  interns 

(5)  residents 


The  purpose  behind  the  interviews  was  to  gather  information 
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on  clerkship  design,  and  to  evaluate  the  students  and 
faculty.  Aside  from  the  interviews,  the  investigators 
studied  and  evaluated  the  patient  records  prepared  by  the 
clerks. 

Unfortunately,  the  ACMC  study  was  terminated  in  the 
early  months  of  1977  before  any  results  were  published.  The 
reasons  for  the  cancellation  were  identified  as  inadequate 
preparation  of  the  study  instrument,  and  poor  study 
response,  leading  to  incomplete  and  erroneous  data 
collection.1  The  ACMC  is  currently  attempting  to  revitalize 
the  study  instrument  for  a  possible  second  study  tentatively 
scheduled  for  1981. 

2.  9  The  Saskatchewan  Psychiatric  Patient  Classification 

Study 

Bay  et  al.  (1974)  designed  a  sampling  procedure  to  be 
used  for  the  Psychiatric  Patient  Classification  Study 
conducted  by  the  Hospital  System  Study  Group,  University  of 
Saskatchewan,  Saskatoon,  Saskatchewan.  The  purpose  of  the 
study  was  to  estimate  direct  patient  care  time.  More 
specifically,  the  study  attempted  to  measure  the  direct  care 
time  provided  by  various  health  professionals  for  the 


1  This  information  was  obtained  by  telephone 
conversations  with  Mrs.  S-  Deschenes,  Association  of 
Canadian  Medical  Colleges. 


£ 

. 


- 

i 

1  m  C 


. 


■ 


I  |HM 


i 


33 


psychiatric  patient  in  an  effort  to  validate  a  patient 
classification  system  developed  by  the  Hospital  System  Study 
Group  (Sjoberg,  K.  and  Bicknell,  P.,  1968).  Rather  than 
discuss  the  study  results,  the  author  will  attempt  to 
highlight  the  significant  aspects  of  the  sampling 
methodology. 

The  sampling  design  was  of  a  multi-stage,  stratified 
cluster  variety,  and  involved  the  selection  of  week-days, 
shifts,  patients,  and  time-segments  for  the  observation  of 
patient  activities  by  the  observers.  Four  hospitals  were 
selected  for  the  study  on  a  non-probability  sampling  basis, 
one  from  each  of  the  four  Western  Canadian  provinces.  Target 
populations  were  defined  for  each  hospital,  and  the  study 
frame  was  set  at  six  consecutive  weeks. 

The  sampling  elements  were  defined  as  segments  of  time 
elapsed  for  each  patient,  or  patient-hours.  The  population 
was  stratified  by  shift  (day,  evening,  and  night) ,  by 
patient-care  category  (minimum,  average,  above  average,  and 
intense) ,  and  by  day  of  the  week.  A  number  of  patient 
observation  periods,  or  patient-shifts,  were  selected  for 
observation,  and  each  patient-shift  constituted  a  cluster, 
consisting  of  patient-hours  over  an  eight-hour  shift. 

Selection  of  the  sample  was  three-stage.  In  the  first 
step,  a  number  of  weeks  out  of  a  possible  six  were  chosen. 
The  second  stage  involved  the  selection  of  patients  from 
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selected  weeks  by  a  simple  random  sampling  technique.  The 
last  stage  of  selection  was  that  of  observation  time  out  of 
a  possible  eight  hours  per  shift.  To  reflect  unequal 
probabilities  of  selection,  weights  were  applied  to  the  data 
at  all  three  sampling  stages  to  obtain  unbiased  estimates  of 
the  population  total,  and  eventually  to  obtain  mean  direct 
care  time  per  patient  shift  for  each  class  of  patients.  For 
selected  time  periods,  both  patients'  activity  and  contact 
with  health  professionals  were  recorded  by  the  independent 
data  collector. 

The  analysis  of  the  data  took  on  two  forms:  (1)  Type  A 
analyses  estimated  the  total  time  spent  by  the  staff,  while 
(2)  Type  B  analyses  were  concerned  with  the  total  time  the 
patients  spent  with  the  staff. 

The  study  design  used  in  this  Phase  III  study  partially 
resembled  that  described  in  this  section.  A  major  difference 
between  the  two  was  that  the  Saskatchewan  study  employed 
outside  independent  observers  for  actual  data  collection, 
while  the  Phase  III  study  relied  on  self-recording  by  the 
students.  A  detailed  description  of  the  Phase  III  sampling 
scheme  will  be  discussed  in  Chapter  3. 

2. 10  Some  Basic  Concents  of  Sampling 

In  this  section,  the  essential  elements  of  sampling 
theory  are  reviewed  in  relation  to  the  present  study.  These 
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can  be  found  in  most  elementary  text  books  on  sampling 
theory,  such  as  Cochran  (1966) ,  Deming  ( 1966) ,  and  Raj  (1972). 


2-10.1  Sampling  Terminology 

Aside  from  those  definitions  found  in  Section  1.8,  the 
following  is  a  list  of  terms  generally  used  in  conjunction 
with  sampling  theory  and  application  (Raj,  1972): 


1.  Population: 

A  group  of  objects,  or  attributes,  such  as 
persons,  establishments,  or  other  items.  A 
population  is  finite  if  the  number  of  objects  or 
units  contained  in  it  is  limited.  Most  of  the 
populations  dealt  with  by  survey  researchers  are 
finite. 

2.  Unit: 

A  unit  is  an  individual  person  in  a  human 
popultion,  a  card  in  a  file  of  cards,  etc.  A 
population  is  built  up  of  elementary  units  which 
cannot  be  further  broken  down. 

3.  Sampling  Unit: 

A  group  of  elements  used  for  the  purposes  of 
selection  of  a  sample,  such  as  a  village  of 
inhabitants,  a  class  of  students,  one  hour  of 
patient  activities,  etc. 

4.  Cluster: 

A  group  of  elementary  units,  such  as  a 
cluster  of  persons  (a  household,  etc.),  which 
constitute  a  sampling  unit. 

5.  Sampling  Fraction: 

The  fraction  of  the  population  selected  for 
the  sample.  The  reciprocal  of  the  sampling 
fraction  is  known  as  the  inflation  factor. 
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6.  Systematic  Sampling: 

The  selection  of  a  sample  from  a  list,  by 
selecting  one  out  of  the  first  k  units,  and 
selecting  every  kth  unit  thereafter. 

7.  Probability  Sample: 

A  sample  in  which  every  sampling  unit  in  the 
population  has  a  predetermined  or  ascertainable 
probability  of  being  selected.  If  no  repetitions 
are  allowed,  it  is  a  probability  sample  selected 
without  replacement.  If  repetitions  are  permitted, 
the  sample  is  selected  with  replacement. 

8.  Probability  Proportionate  to  Size: 

When  the  units  appear  in  the  sample  with 
probabilities  proportional  to  some  measure  of  size 
of  the  units  [ ie.  the  number  of  elements  in  a 
unit  ]. 

9.  Strata: 

When  the  population  is  divided  up  into  groups 
and  a  sample  is  selected  from  each  group,  the 
groups  are  called  strata  [pp.  11-12]. 


2. 10.2  Characteristics  and  Advantages  of  Sampling  Methods 

Sampling  is  freguently-used  technique  for  gathering 
information,  and  can  be  of  great  value  "to  summarize,  for 
the  guidance  of  administrators,  facts  which  would  otherwise 
be  inaccessible  owing  to  the  remoteness  or  obscurity  of  the 
units  involved  or  there  numerousness  [Raj,  1972,  p. xv]".  In 
the  case  of  the  present  investigation,  "the  facts"  consist 
of  estimates  of  time  spent  in  contact  between  Phase  III 
students  and  medical  faculty  members. 
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Deming(1966)  related  six  characteristics  and  advantages 
of  sampling  over  other  statistical  procedures: 


(1)  Usefulness  and  comprehensiveness  of  content, 

(2)  Reliability  of  results,  sufficient  for  the 

purpose, 

(3)  Intellegiblity  (classifications  and 

definitions  that  are  understood)  , 

(4)  [Greater]  speed, 

(5)  [Increased]  economy  of  operation,  and 

(6)  Accurate  interpretation  and  presentation 

[pp.  3-4]. 


2 . 10.3  Stratification  and  Clustering 1 


A  technique  often  used  in  sample  surveys  is 
stratification.  Stratification  is  defined  as  a 
classification  or  separation  of  elements  into  a  number  of 
mutually  exclusive  and  exhaustive  sub-populations,  or 
strata.  This  technique,  if  properly  carried  out,  is 
advantageous  (1)  in  reducing  error  due  to  sampling  and 
consequently  increasing  efficiency,  (2)  in  enabling  sampling 
of  rare  events,  (3)  in  enabling  sampling  by  various 
procedures  over  different  sub- populations,  (4)  in  decreasing 
administrative  effort,  and  (5)  in  enabling  independent 
estimation  at  each  stratum  level. 


i  The  information  reviewed  in  this  section  can  be  found 
in  any  standard  text  on  sampling  theory,  such  as 
Cochran  (1966)  ,  Deming  ( 1966)  ,  and  Raj  (1972). 
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Another  frequently-used  technique  is  clustering. 
Clustering  is  a  method  whereby  several  elements  are  grouped 
together  to  form  a  new,  single  sampling  unit.  The  main 
purpose  behind  clustering  is  to  reduce  the  cost  of  data 
collection. 

In  order  to  avoid  subjective  biases,  a  probability 
selection  method  is  almost  always  desired,  and,  to  allow 
"probability  selection",  the  theory  of  probability  is 
applied.  Probability  sampling  assumes  that  every  sampling 
unit  in  the  frame  has  a  pre-determined  or  ascertainable  and 
non-zero  probability  of  being  selected.  However,  it  does  not 
necessarily  assume  equality  of  selection  probability. 

2.10.4  Sampling  Errors 1 

There  are  three  major  types  of  uncertainties  commonly 
found  in  sample  surveys.  Structural  limitations  [or  the  so- 
called  Type  I  errors]  are  independent  of  the  size  or  kind  of 
sample  being  used.  Examples  of  Type  I  errors  are:  (1) 
measurement  deficiencies,  inneffective  coding  rules,  (2) 
ommissions  of  certain  classes  of  the  universe,  poor  timing 


i  This  section  relies  on  basic  information  found  in 
Deming ( 1 966) . 
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of  the  survey,  (3)  bias  due  to  misuse  of  weights,  and  (4) 
unwarranted  interpretations  of  the  results,  failure  to 
report  limitations  of  the  study. 

Operational  [or  Type  II]  errors  are  usually  single, 
specific  mistakes  that  occur  while  one  is  conducting  a 
survey.  Type  II  errors  may  be  caused  by  lack  of  quality 
control  or  by  mistakes  made  by  interviewers. 

Sampling  [Type  III]  errors  are  defined  as  errors  due  to 
(1)  selections  of  samples,  or  (2)  non-systematic  measurement 
errors.  These  errors  are  inherent,  uncorrelated,  non- 
persistent,  accidental,  and  are  of  a  cancelling  nature. 
However,  sampling  errors  are  the  only  ones  which  can  be 
partly  estimated,  and  this  may  be  accomplished  by 
calculation  from  the  sample  itself,  under  certain 
conditions. 

Since  the  sample  uses  a  portion  of  the  population  to 
estimate  the  population  characteristics,  estimates  based  on 
sampling  methodologies  are  subject  to  "sampling  error". 

Yates  (1965)  tells  us  that  "even  if  a  proper  process  of 
selection  is  employed,  the  sample  cannot  be  exactly 
representative  of  the  whole  aggregate...  errors  will  depend 
on  the  size  of  the  sample,  on  the  variability  of  the 
material,  on  the  sampling  procedure  adopted,  and  on  the  way 
in  which  the  results  are  calculated  [p-2]." 
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To  reiterate,  sample  survey  estimates  are  subject  to 
two  major  classifications  of  errors  —  sampling  and  non¬ 
sampling,  Gonzalez  (1975)  states  that  sampling  errors  result 
from  observations  made  only  on  a  sample  rather  than  the 
entire  population.  He  further  explains  that  non-sampling 
errors  can  be  attributed  to  sources  such  as: 

(1)  the  inability  to  obtain  information  from  all 

cases  in  the  sample 

(2)  the  difficulties  in  defining  terms  and 

assumptions 

(3)  the  variation  in  the  interpretation  of  survey 

questions  by  the  respondents 

(4)  the  inability  or  unwillingness  of  respondents 

to  provide  correct  information 

(5)  operational  survey  errors  (ie.  in  the 

collection,  editing,  coding  and 

processing  of  survey  data) 

Sampling  deviation,  as  defined  by  Gonzalez,  is  the 
difference  between  a  sample  estimate  and  the  average  of  all 
possible  sample  estimates.  Standard  error  of  a  sample 
estimate  is  a  measure  of  the  variation  among  estimates  from 
all  possible  samples,  and  is  thus  a  measure  of  the  precision 
with  which  an  estimate  from  a  particular  sample  approximates 
the  average  result  of  all  possible  samples.  The  relative 
standard  error  of  a  sample,  or  the  coefficient  of  variation, 
is  defined  as  the  standard  error  of  the  estimate  divided  by 
the  value  being  estimated.  The  standard  error  partially 
measures  the  effect  of  certain  non-sampling  errors  as  well. 
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but  does  not  measure  any  systematic  biases  in  the  data.  Bias 
is  the  difference  between  the  sample  estimate  and  the 
desired  value,  averaged  over  all  possible  samples. 

2. 1 1  Summary 

Operations  research,  or  the  application  of  the 
scientific  method  to  problems  involving  the  operations  of 
systems,  has  had  widespread  application  in  all  facets  of 
both  private  and  public  management.  As  the  level  of 
technology  and  the  complexities  of  management  increase,  so 
does  the  need  for  study  into  the  ways  and  means  of  improving 
efficiency  and  effectiveness.  Quite  often,  operations 
research  has  proven  to  be  an  effective  cost- reduction  and 
output-maximization  technique. 

In  the  health  sector,  operations  research  has  been 
frequently  utilized  in  recent  years,  particularly  in 
hospitals.  Lately,  however,  its  methodologies  have  been 
challenged  as  being  too  mechanistic,  and  neglecting  such 
subjective  measures  as  the  quality  of  patient  care. 

Medical  educators  have  called  for  an  increase  in 
system-oriented  research.  The  costs  of  training  physicians 
is  on  a  steady  increase,  and  standards  for  licensure  are 
constantly  being  re-evaluated  in  light  of  new  technological 
advancements  in  medical  science,  and  the  introduction  of  new 
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curricular  programs. 

Studies  evaluating  the  many  facets  of  medical  education 
have  been  numerous  and  widespread,  however  manpower  studies 
have  been  scarce.  Courses  and  instructors  are  being 
evaluated  on  a  regular  basis,  and  a  study  is  underway  to 
establish  a  medical  school  information  system  to  improve 
reporting,  operation,  and  budgetary  control. 

Two  studies  have  been  conducted  to  assess  the  level  of 
contact  between  clinical  clerkship  students  and  their 
supervisory  staff  and  instructors.  Results  appeared  useful 
in  estimating  resource  distributions  amongst  education  and 
patient-care,  as  well  as  in  evaluating  the  clerkship 
programs  themselves  and  relating  their  purpose  to  the 
overall  philosophy  of  the  Canadian  medical  schools. 

The  terminology  of  sampling  theory  was  defined,  and  the 
characteristics  and  advantages  of  the  various  sampling 
methods  were  reviewed. 


. 
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Chapter  3 
METHODOLOGY 


3. 1  Research  Strategies 

As  outlined  in  Chapter  1 ,  the  primary  objective  for 
this  study  was  to  establish  an  accurate  estimation  of  Phase 
III  faculty-student  contact  time  and  to  evaluate  the 
methodologies  applied.  In  other  words,  the  task  was  to 
estimate  the  time  spent  by  various  members  of  the  Faculty  of 
Medicine  in  direct  contact  with  Phase  III  medical  students. 

In  order  to  facilitate  the  most  appropriate 
methodologies  of  data  collection,  several  alternative  modes 
were  considered.  The  author  was  faced  with  three  primary 
questions  to  be  resolved: 

(1)  From  whom  was  the  data  to  be  collected? 

(2)  What  was  to  be  the  time  frame  of  the  data 

collection  phase? 

(3)  How  was  the  data  to  be  collected? 
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3. 1 • 1  Survey  Respondents 

In  deciding  who  was  to  record  the  faculty-student 
contact  times  and  report  to  the  researchers,  there  were 
several  alternatives  considered.  Grady  et  al.  (1975) ,  in 
their  research  proposal,  considered  four: 

(1)  faculty  preceptors; 

(2)  paid  observers; 

(3)  paid  interviewers;  and 

(4)  the  students  themselves. 

Alternatives  (1)  and  (2)  were  ruled  out  immediately  by  these 
authors,  primarily  due  to  previous  experiences  by  others 
which  showed  inadequacies  of  the  methods  and  overburdening 
expense.  Alternative  (3)  was  also  rejected,  due  to  the 
potential  threat  to  and  annoyance  for  the  students  and 
preceptors,  as  well  as  the  difficulty  in  finding 
interviewers  to  work  one  hour  each  evening  for  a  year.  The 
last  alternative  (4)  was  deemed  most  feasible,  mainly 
attributed  to  conveniences  such  as: 

(1)  the  lack  of  need  for  observers  or 

interviewers,  thereby  avoiding  annoyance 
they  might  create, 

(2)  a  reasonable  degree  of  objectivity,  and 

(3)  the  reasonable  cost  in  carrying  out  such  a 

study. 
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In  selecting  alternative  (4)  ,  the  authors  did  take  into 
account  the  potential  disadvantages.  For  example,  student 
respondents  were  potentially  biased,  reflecting  more  the 
students*  point  of  view  rather  than  the  actual  amount  of 
contact  time.  In  addition,  the  probability  of  non-response 
problems  was  increased,  due  tc  the  students*  busy  schedules, 
and  lack  of  incentives  or  motivation.  However,  the  Grady  et. 
al. (1975)  ascertained  that  student  respondents  still 
constituted  the  most  feasible  alternative. 

3.1.2  Time  Frame  of  the  Study 

The  alternatives  considered  by  Grady  et  al.  (1975)  for 
the  time  frame  were  (1)  a  cross-sectional,  one-week  frame, 
and  (2)  a  frame  over  a  period  of  time.  The  cross-sectional 
frame,  or  the  sampling  of  a  population  once,  certainly  would 
have  been  the  simplest  choice.  However,  as  suggested  in  the 
ACMC  Study  discussed  in  Chapter  2,  this  method  fails  to 
account  for  variations  over  time,  and  inferences  can  only  be 
made  with  respect  to  the  particular  time  period  chosen  for 
sampling. 

On  the  other  hand,  a  substantially  long  time  frame 
reduces  levels  of  uncertainty,  and  more  accurate  results  can 
thus  be  obtained.  It  must  be  noted,  however,  that  a  system 
can  undergo  characteristic  change  over  such  a  time  period, 
and  this  variation  cannot  be  systematically  identified. 
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Nevertheless,  the  one-year  time  frame  chosen  by  Grady  et 
al.  (1975)  was  accepted  because  it  coincided  with  the 
existing  academic  cycle.  In  addition,  this  frame  enabled  the 
cancelling  out  of  variations  due  to  seasonality,  and  over  a 
year's  period,  the  academic  cycle  remains  relatively  stable 
(ie.  no  course  content  alterations) .  In  terms  of  resource 
allocation,  a  one-year  time  frame  is  also  well-matched  to 
the  cycle  of  the  Faculty's  annual  budget. 

3.1.3  Data  Collection  Method 


Grady  et  al.(1975)  considered  the  question  of  "how  the 
data  is  to  be  collected”  and  reduced  it  to  two 
possibilities:  by  complete  enumeration  or  by  sampling. 

Raj  (1972)  points  out  that  sampling  can  be  as  dependable  a 
method  as  that  of  complete  enumeration  or  census,  for  the 
following  reasons: 


(1)  Census  data  is  subject  to  serious  errors  due 

to  the  need  for  a  multitude  of  qualified 
enumerators,  tabulators,  and 
supervisors,  of  whom  there  are  limited 
quantities  available. 

(2)  Large  scale  censuses  are  characterized  by 

large  systematic  errors;  sample  surveys 
are  small  and  more-qualified  workers  are 
available  that  can  be  adequately 
trained. 

(3)  The  cost  of  a  sample  survey  is  usually  a  mere 

fraction  of  that  of  a  census,  and 
results  can  be  produced  considerably 
faster. 

(4)  The  scope  of  inquiry  can  be  broader  by  using 
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sampling  methods,  due  to  the  possibility 
of  collecting  information  of  a 
specialized  type  or  of  a  more  delicate 
nature  [ pp. 7-8  ] . 

Although  there  are  some  limitations  in  the  use  of  a  sample 
survey,  this  method  was  chosen  for  the  present  study.  The 
rationale  for  using  the  sample  survey  is  discussed  in 
greater  depth  later  in  this  chapter. 

To  reiterate,  the  information  obtained  from  the 
research  proposal  (Grady  et  al. ,  1975)  led  this  author  to 
select  the  following  methodological  framework:  (1)  a  sample 
survey,  (2)  to  be  carried  out  over  an  extended  period  of 
time,  and  (3)  using  the  Phase  III  students  as  respondents. 

3. 2  Sampling  Design 

A  stratified,  cluster  sampling  design  was  developed  for 
the  Phase  III  study.  The  following  sections  outline  the 
details  of  this  design. 

3. 2. 1  Assumptions  and  Constraints 

Like  any  other  sampling  survey,  the  actual  design  is 
influenced  by  environmental  and  resource  constraints.  The 
following  are  the  major  assumptions  and  constraints  that 
applied  to  the  present  study. 
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1.  Variability  of  Contact  Time: 

It  was  assumed  that  the  variablilty  of 
contact  time  within  any  hospital-department- 
rotation  week  unit  was  less  than  the  variation  of 
contact  time  amongst  these  units.  Stratification 
is  performed  to  take  advangage  of  this  assumption. 

2.  Sample  Size: 

It  was  postulated  that  too  many  requests 
would  decrease  the  students'  interest  in  the 
study,  resulting  in  adverse  effects  on  the 
response  rate  and  quality  of  data.  It  was  assumed, 
based  on  the  results  of  the  pilot  study,  that 
students  could  be  asked  to  complete  the  instrument 
an  average  of  four  times  each,  over  the  52-week 
time  frame,  without  placing  too  much  pressure  on 
them. 

3.  Cyclical  Homogeneity: 

It  was  assumed  that,  within  a  one-year 
period,  student  activity  is  more  or  less  similar 
over  the  same  rotation- week.  In  other  words. 
Rotation-week  # 1  of  the  Surgery  department  in 
September  would  generate  activities  similar  to 
Rotation-week  #1  of  Surgery  in  May  at  the  same 
hospital.  This  assumption  also  covers  changes  of 
preceptors  (or  other  staff)  within  a  single 
hospital  and  department  during  the  year- 

4.  Homogeneity  Within  Cohorts: 

Within  a  single  cohort,  it  was  assumed  that 
the  students'  activities  would  be  more  or  less  the 
same  for  a  given  rotation  week.  This  infers  that 
at  a  particular  hospital,  in  a  particular 
department,  and  during  a  particular  rotation  week, 
all  students  perform  relatively  homogenous 
activities.  However,  activities  may  vary 
substantially  over  different  rotation  weeks. 


3.2.2  Design  Specification 


The  following  design  specifications  were  applied  to  the 
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Phase  III  study: 


1.  Elementary  Unit: 

The  basic  unit  for  this  study  was  defined  as 
one  week  of  student  activity  [a  student-week ]r 
beginning  on  Thursday  at  9:00  A.M.,  and  ending  the 
following  Thursday  at  9:00  A.M.  The  reason  for  the 
selection  of  a  student- week  as  the  basic  element 
was  because  (1)  a  week  constitued  a  natural  unit 
in  the  academic  cycle,  and  (2)  a  week  was  a 
reasonable  span  of  time  for  a  student  to  remember 
his/her  activities  (Grady  et  al.,  1975). 

2.  Study  Population: 

The  study  (or  accessable)  population  was  all 
Phase  III  student-weeks  between  September  4,  1975, 
and  September  3,  1976  at  the  University  of  Alberta 
Faculty  of  Medicine.  It  was  the  intention  of  the 
author  to  derive  possible  generalizations  for 
future  years,  based  upon  results  obtained  from  the 
study  population.  This  includes  a  full  year  of 
academic  activities  for  Phase  III  students.  Since 
Phase  III  consists  of  relatively  equal  numbers  of 
third-  and  fourth-year  students,  the  study 
population  may  be  considered  to  represent  the 
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activity  of  two  years  of  the  program  for  each 
student.  At  the  design  stage,  the  total  number 
(N.j.)  of  student-weeks  was  estimated  by: 


Nt=  108  X  (52  +  28)  X  2  X  1/2 
=  8 6 <4 0  student-weeks 


since : 


r* - - - - - * - — — - 

1  Equation  term  definition: 


I 

4 


- — n 

Value  | 

-H 


Average  number  of  Phase  III 

student  intern  positions/year 


108 


Formal  rotation- weeks/year  | 

required  for  Phase  III  students 


52 


j  Elective-weeks/year  I 

|  required  for  Phase  III  students 

I  I 

I  I 

1  Phase  III  (3rd  and  4th)  years  | 

i  I 

|  Study  years  per  Phase  III  year  | 

(  I 

L - JL 


28 

2 

1/2 


The  basis  behind  the  determination  of 
population  size  results  from  the  fact  that  a  Phase 
III  student  is  expected  to  complete  his/her  third 
and  fourth  years  of  the  program  over  a  two-year 
period.  In  light  of  this,  the  study  population  was 
estimated  at  8640  student- weeks.  However,  due  to 


some  uncertainty  of  the  number  of  students  in  each 
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year,  actual  population  size  varied  somewhat. 

3.  Sample  Size: 

The  number  of  elements  (n  )  chosen  to 

T 

represent  the  Study  Population  was  determined  by: 


nT  =  2  X  108  X  4 
-  864  student-weeks 


since: 


I 


Equation  term  definition: 


T - - "1 

J  Value  | 
1 - H 


1  Number  of  Phase  III  years  j 

|  Number  of  student  intern  j 

J  positions/year  1 

|  Average  number  of  sampled  1 

|  weeks/st udent  J 

« _ _ _ _ _ ...... - - - - - - L 
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108  | 

4  | 
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Assumption  2  states  that  students  were  deemed 
able  to  complete  an  average  of  four  instruments 
over  the  study  period.  Under  this  constraint, 
then,  the  total  sample  size  was  set  at  864 
st udent- weeks,  giving  an  overall  sampling  ratio 


[ n : N ]  of  1:10. 


4.  Stratification: 


The  population  was  stratified  over  four 
levels  (1)  for  administra tative  convenience,  (2) 
judged  to  be  natural  divisions,  and  (3)  to  account 
for  homogenous  student  activity.  The  first  level 
of  stratification  was  by  type  of  internship 
[either  formal  rotation  or  elective].  The  elective 
strata  were  further  broken  down  into  52  calendar- 
weeks,  with  a  total  (N^-)  of: 


Ke  =  28  X  108  X  2  X  1/2 
=  3024  student-weeks 


Therefore,  the  sub-population  of  the  elective 
stratum  came  to  a  total  of  3024  student-weeks. 

The  formal  rotation  stratum  consisted  of  a 
total  (Nr)  of: 

N  =  52  X  108  X  2  X  1/2 
R 


5616  student-weeks 
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The  total  population  size  of  the  rotation 
stratum  was  5616  student- weeks. 

The  formal  rotation  stratum  was  further 
broken  down  by  hospital,  by  department,  and  by 
rotation- week. 

By  utilizing  a  stratification  scheme,  the 
author  attempted  to  isolate  systematic  differences 
in  responses  characteristic  to  each  particular 
stratum,  and  subsequently  minimize  standard  error. 

The  stratification  scheme  is  diagramatically 
represented  in  Figure  1. 

In  summary  then,  the  sampling  design  was  both  cross- 
sectional  and  longitudinal,  and  was  time-dependent, 
accounting  for  changes  of  the  system  over  the  52-week  time 
frame.  A  multi-level  stratification  scheme  was  also 
developed  to  minimize  overall  standard  error. 

3. 3  Sample  Allocation  and  Selection 

Allocating  a  sample  to  the  various  strata  can  be 
handled  in  various  ways  (Raj,  1972,  p.  51).  One  method, 
known  as  the  “proportional  allocation  technique”,  dictates 
that  the  sample  size  in  a  stratum  is  proportional  to  the 
total  number  of  units  in  that  stratum.  For  an  optitmum 
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LEVELS  OF  STRATIFICATION 
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allocation,  the  Neyman  Allocation  Formula  (Raj,  1972,  p.51) 
indicates  that  the  number  of  units  in  the  sample  from  a 
stratum  should  be  proportional  to  the  product  of  the  stratum 
size  and  the  stratum  standard  deviation,  and  inversely 
proportional  to  the  square  root  of  the  unit  cost.  However, 
the  Neyman  formula  is  not  directly  applicable  to  this  study 
because  (1)  the  stratum  standard  deviations  are  unknown,  and 
(2)  because  of  the  need  for  a  breakdown  into  small  strata.  A 
third  method,  that  of  "equal  allocation",  is  conducted  such 
that  the  same  number  of  units  are  selected  from  each  final 
stage  stratum  irrespective  of  the  size  of  the  stratum. 

In  selecting  and  allocating  the  stratified  sample  for 
the  Phase  III  study,  an  equal  allocation  rule  was  employed. 
This  method  was  selected  because  of  its  simplicity,  and 
because  of  of  the  facility  of  utilizing  many  small  strata, 
each  containing  sufficient  numbers  of  units  to  enable 
statistical  analysis.  Although  the  original  design  included 
a  three-way  replication,  non-response  problems  reduced  the 
number  of  elements  in  each  stratum  to  a  level  prohibiting 
the  use  of  statistical  analysis.1  As  a  result,  the 


1  The  problem  of  non-response  was  not  identified  until 
the  data  collection  phase  of  the  study  was  well  underway. 
Therefore,  although  the  sampling  was  carried  out  as  if  a 
three-way  replication  was  in  force,  the  data  was  ultimately 
aggregated  to  eliminate  the  replication  factor.  This  is 
discussed  in  more  detail  in  Chapter  4.1. 
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replication  factor  was  ultimately  discarded  from  the  study. 

Because  the  methods  of  selection  differed  for  formal 
rotation  and  elective  strata,  they  are  discussed  separately. 

3. 3. 1  Formal  Rotation  Selection 

The  formal  rotation  stratum  was  broken  down  by 
hospital,  by  department,  and  by  rotation  week,  and  was 
selected  in  the  following  manner. 

For  each  of  the  25  hospital-department  combinations 
available  to  Phase  III  students  (see  Table  I) ,  a  chart  was 
prepared  in  a  similar  format  to  that  illustrated  in  Figure 
2.  This  particular  example  chart  was  used  to  select  students 
from  the  Medicine  (Part  1)  department  at  the  Royal  Alexandra 
Hospital. 

In  this  example,  the  rotation  was  of  eight  weeks 
duration  (rotation  weeks  are  listed  down  the  left-hand 
column) ,  and  six  consecutive  rotations  (or  cohorts)  were 
sampled  over  the  one-year  study  time  frame,  giving  a  total 
of  48  calendar  weeks  (Calendar  week  #17  was  eliminated 
because  of  the  Christmas  vacation;  therefore,  a  total  of  49 
calendar  weeks  actually  appear  on  the  chart) .  For  each 
hospital-department  group  analyzed  in  this  manner,  the 
number  of  cohorts  was  pre-determined ,  based  on  schedules 
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provided  by  the  Office  of  the  Dean  of  Medicine,  by  the 
number  of  times  the  duration  of  the  rotation  (12  weeks 
rotation  for  Surgery,  8  for  the  rest)  could  be  divided 
evenly  into  the  52  weeks  of  the  study  time  frame. 

The  first  stage  of  selection  involved  the  selection  of 
three  cohorts  from  each  of  the  eight  rotation  weeks.  These 
cohorts  of  student  weeks  were  known  as  clusters  of  sampling 
elements.  The  above  selection  was  done  by  simple  random 
sampling,  without  replacement.  The  three  selected  cohorts 
for  Rotation  Week  #1  in  the  example  chart,  were  Cohorts  1, 

4,  and  6  respectively  (A) .  The  appropriate  columns  were  then 
marked  with  the  assigned  replication  number  (B) . 

The  next  stage  in  the  process  consisted  of  the 
selection  of  student-weeks  from  each  of  the  previously- 
selected  cohorts.  In  the  example.  Rotation  Weeks  1,  4,  and  6 
had  already  been  selected  from  Cohort  #1  by  the  first-stage 
selection  process,  giving  a  sample  size  of  three  student- 
weeks  (C) .  It  was  also  known  in  advance  that  six  students 
were  registered  for  the  first  8-week  rotation  (D) . 

Therefore,  by  simple  random  sampling  without  replacement, 
three  student-weeks  were  selected  from  a  possible  six,  with 
equal  selection  probability  (E) .  When  the  sample  size 
exceeded  the  number  of  registered  students,  as  it  did  in 
Cohort  #6,  repetitions  were  permitted. 
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The  number  of  students  registered  in  each  rotation  was 
obtained  from  lists  of  the  students'  names,  made  available 
to  the  author  by  the  Faculty  of  Medicine.  The  students' 
names  appeared  on  the  lists  in  random  sequence.  The  final 
selection  of  student  weeks  was  made  by  applying  the  random 
numbers  obtained  in  (E)  to  the  randomly-ordered  lists  of 
students.  For  example,  the  illustration  chart  in  Figure  3.2 
shows  random  numbers  of  6,  2,  and  1  selected  for  Cohort  #1. 
This  means  that  if  John  Smith  was  the  sixth  name  on  the 
list,  he  would  be  selected  to  respond  during  Rotation  Week 
#1  and  Calendar  Week  #1.  Similarly,  the  person  whose  name 
appeared  second  on  the  list  would  be  a  respondent  during 
Rotation  Week  #4  and  Calendar  Week  #4.  This  procedure  was 
carried  out  for  the  entire  chart,  and  it  can  be  shown  that 
the  third  person  on  the  sixth  rotation  list  was  chosen  to 
respond  during  Rotation  Week  #8,  Calendar  Week  #49  (F)  . 

3. 3. 2  Elective  Selection 

The  elective  student-weeks  were  stratified  into  fifty- 
two  calendar-week  groups.  Students  assigned  to  electives 
were  selected  by  a  special  systematic  random  sampling 
technique.  To  illustrate,  lists  of  students  were  obtained 
every  three  months  showing  the  electives  to  which  each 
student  was  assigned.  The  list  was  ordered  so  that  the 
students'  names  appeared  in  the  order  of  the  commencement 
date  of  their  chosen  electives.  Student- weeks  were  then 
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selected  randomly  [and  with  equal  probability]  from  the  list 
with  a  selection  ratio  of  1: 10-  Each  of  the  selected 
student-weeks  was  sequentially  assiqned  a  replication  number 
[A,  B,  or  C],  giving  an  overall  selection  ratio  of  1:30 
(one-tenth  X  one-third) . 

3-4  Assignment  of  Weights 

Due  to  the  unequal  stratum  population  sizes,  and 
subsequent  unequal  probabilities  of  selection,  an  assignment 
of  weights  was  necessary  to  adjust  the  results  in  order  to 
achieve  the  representation  of  the  various  strata  in  the 
total  population;  that  is,  to  provide  unbiased  estimates. 

This  can  be  shown  in  the  following  manner.  Assuming 
that  the  probability  of  selecting  a  sampling  element  i  (with 
measurement  value  Yi)  is  given  by  Pi,  and  assigning  the 
subject's  weight  Wi,  then  the  weighted  element  Zi  is  defined 
as  the  product  of  Wi  and  Yi.  The  total  population  estimate 
(ty)  can  then  be  expressed  as  £y  =  E  Zi  =  n-2,  where  n  is  the 
total  sample  size  and  Z  is  the  sample  mean  of  weighted 
values.  The  mean  estimate  (£y)  can  be  related  to  population 
size  N  by  yy  =  £y/N  and  also  y y  =  nZ/N.  If  Wi  is  inversely 
proportional  to  the  selection  probability  Pi,  it  can  be 
shown  that  E  (  ty)  =  Cy  and  E(yy)  =  vy,  where  E  denotes 
statistical  expectation  (Raj,  1972).  The  above  equalities  of 
expected  values  with  population  values  signify  unbiased 
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estimates. 

The  requirement  for  unbiased  estimation  is  that  the 
weight  must  be  inversely  proportional  to  the  selection 
probability,  namely: 

W.  a  1/P 
i  i 

If  egual  probabilities  are  used  for  a  stratum  h,  then 
the  selection  probabilities  and  weights  are  common  to  all 
units  selected  from  the  stratum,  namely  P  =n  /N  ,  and 

Hh  a  Nh//n  h*  By  talcin9  a  constant  of  proportionality  equal 
to  one,  then: 


Nh/" 


h 


where:  W ^  =  the  assigned  weight  for  Stratum  h 

n^  =  sample  size  of  Stratum  h 
N  ^  =  stratum  population  size. 

However,  this  formula  assumes  perfect  response  rate,  and  as 
noted  earlier,  the  rate  of  response  achieved  by  the  study 
was  relatively  poor.  This  being  consistent  with  most  studies 
relying  on  the  mail  questionnaire  method,  a  second  weighting 
factor  was  applied  to  the  above  expression  to  account  for 
missing  data  [non-response]  within  each  stratum.  Underlying 
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this  adjustment  is  the  assumption  that,  within  each  stratum, 
the  non-respondents  carried  out  more  or  less  the  same 
activity  as  the  respondents. 

Therefore,  if: 

®  h  =  tiie  #  of  valid  samples  in  Stratum  h 


then, 

w  h  =  Nh  /nh  x  nh/mh  '  or 


Hh  = 


In  calculating  the  weights  for  the  Phase  III  study 
sample,  the  author  employed  the  following  procedures: 

(1)  Calculation  of  W_1  f  equal  to  /n 

For  each  of  the  formal  rotation  strata,  the  population 
size  [  student- weeks  ]  was  calculated  by  multiplying  the 

number  of  cohorts  in  the  stratum  by  the  number  of  students 
participating  in  the  rotation.  The  stratum  sample  size  [n  ] 
always  had  the  value  three  [the  number  of  selected  cohorts 
per  stratum]  because  of  the  equal  allocation.  Therefore,  the 
value  of  W 1  for  each  stratum  [h]  was  given  by: 
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in  h  =  #  of  student  weeks  /  3 

Because  the  elective  sample  was  selected  systematically 
with  a  sampling  ratio  of  1:10,  the  W1  weight  was  heia 
constant  across  all  elective  strata  and  given  a  value: 

W1h  =10/1 


(2)  Calculation  of  f  equal  to 

Due  to  the  low  response  rate,  the  author  was  faced  with 
the  problem  of  "empty-cells",  whereby  the  number  of 
responses  in  a  particular  stratum  [m^]  equalled  zero.  In 
this  situation,  a  weight  [W2]  could  not  be  calculated,  as 
the  expression  [n^/m^]  had  a  zero-denominator.  Because  of 
this  problem,  the  fifty-two  elective  strata  had  to  be 
regrouped  into  thirteen  four-week  strata,  for  weighting 
purposes. 

Once  all  regroupings  were  completed,  K2  was  derived  for 
each  of  the  twenty-five  formal  rotation  and  thirteen 
elective  strata  [h]  by: 

»2h  =  nh  /  “h 


l'he  weights  were  then  applied  to  the  various  variables 
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using  the  "WEIGHT”  facility  of  the  "Statistical  package  for 
the  social  sciences  (SPSS)"  (Nier  1970). 

Tables  I  and  II  present  a  summary  of  weights  applied  to 
the  study  sample.  For  example,  the  total  stratum  weight  for 
rotations  at  the  Royal  Alexandra  Hospital,  department  of 
Obstetrics,  was  17.5.  In  addition,  the  first  of  the  elective 
strata  was  assigned  a  weight  of  39.0. 

3.5  Research  Instruments  and  Selection  Procedures 

The  primary  instrument  consisted  of  a  time  sheet  [see 
Appendix  B].  Every  selected  student  was  asked  to  complete  a 
time  sheet  over  the  weekly  time-period  assigned  to  him/her 
[a  student-week].  The  time  sheet  questions  required 
responses  relating  to  attributes  for  every  activity  engaging 
the  student's  time  during  that  week.  The  attributes  were: 

(1)  activity  type,  (2)  the  number  of  preceptors,  residents, 
interns  and  other  Phase  III  students  present  during  the 
activity,  (3)  the  duration  of  direct  contact  between  the 
student  and  those  listed  above,  and  (4)  an  estimate  of  the 
proportion  of  direct  instruction  perceived  by  the  student 
during  the  activity.  When  the  number  of  faculty  changed 
during  the  course  of  an  activity,  students  were  requested  to 
give  an  average  of  the  number,  rounded  upwards. 


At  the  time  of  preparing  the  sampling  design,  exact 
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Summary  for  Rotation  Samples 
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Table  II 

Summary  for  Elective  Samples 
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student  scheduling  was  unknown,  and  the  author  used  the 
1974/1975  schedule  for  design  purposes.  Approximately  three 
months  in  advance  of  student  selection,  the  finalized 
rotation  and  elective  schedules  of  students1  names  were  made 
available  to  the  study  investigators  by  the  Dean's  Office. 
Immediately  upon  receipt  of  these  schedules,  sampling 
procedures  were  carried  out,  according  to  the  specifications 
outlined  in  Section  3.3,  for  the  three-month  period  directly 
following.  This  "block-type"  format  was  used  to  ensure  up- 
to-date  student  allocations  to  rotations  and  electives 
within  the  various  hospitals  and  departments. 

On  the  Monday  of  every  week,  time  sheets  were  prepared 
for  the  student-weeks  selected,  corresponding  to  the 
calendar-week  immediately  following,  and  "packages"  were 
assembled  and  mailed  to  the  students  the  following  Thursday. 
The  lag-time  between  the  mailing  of  the  time  sheets  and  the 
commencement  of  the  appropriate  calendar-week  was  set  at 
seven  days.  This  was  done  to  (1)  assure  that  the  package 
arrived  at  the  student's  residence  before  the  start  of  the 
current  calendar-week,  and  (2)  to  assure  that  the  package 
did  not  arrive  too  early,  reducing  the  possibility  of  the 
student  forgetting  to  complete  it. 

Returned  time  sheets  were  logged-in  on  two  lists:  one 
indexed  by  calendar— week,  and  the  other  indexed  by 
identification  number.  This  procedure  allowed  for  on-going 
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"response-rates”  to  be  observed  by  the  author  at  a  glance. 

Two  other  auxilliary  instruments  were  utilized  in 
collecting  "follow-up"  information.  Interviews  were 
conducted  with  the  preceptors  of  each  hospital  department 
participating  in  the  Phase  III  program,  for  the  purpose  of 
ascertaining  the  shortcomings,  if  any,  of  the  rotation  or 
elective  programs.  The  other  instrument  took  the  form  of  a 
follow-up  questionnaire,  and  was  sent  to  all  the  Phase  III 
students  who  had  ever  received  a  time  sheet  and  were  still 
enrolled  in  the  Faculty.  This  questionnaire  provided  the 
investigators  with  the  students1  comments  and  criticisms 
pertaining  to  the  content  and  procedure  used  in  the  study 
data^collection  phase. 

3.6  Estimation  of  Margin  of  Errors:  Standard  Error 

Confidence  intervals  may  be  constructed  given  a  sample 
estimate  and  an  estimate  of  standard  error.  For  a  given 
sampling  rate,  and  for  a  prescribed  level  of  confidence,  the 
confidence  interval  includes  the  average  result  of  all 
possible  samples.  As  an  example,  consider  the  case  where  all 
possible  samples  are  selected,  and  from  each  of  which 
calculations  of  estimates  and  estimated  standard  error  are 
made.  It  can  be  said,  then,  that  19  out  of  every  20 
intervals  from  approximately  two  standard  errors  below  the 
estimate  to  approximately  two  standard  errors  above  the 
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estimate  contains  the  average  value  of  all  possible  samples. 
This  is  known  as  a  95%  confidence  interval. 

The  variance  of  a  simple  random  sample  without 
replacement  in  each  stratum  can  be  expressed  as: 

VAR(Cy)  =  Inh  (1-fh)s2h 

where : 

n^  =  the  sample  size  of  stratum  h,  f,  =  the 
sampling  fraction  of  stratum  h,  and  =  the 

sample  variance  of  weighted  attributes  in  stratum 

h. 

The  estimated  standard  error  can  be  expressed  as  the  square 
root  of  the  sample  variance. 

3.7  Summary 

A  stratified,,  cluster  sample  was  selected,  with  a 
sampling  ratio  of  ten  percent.  The  sampling  elements 
[student-weeks]  were  divided  into  two  categories,  namely 
formal  rotations  and  electives,  and  each  category  was 
further  sub-divided  into  meaningful  strata. 

Student-weeks  for  rotations  were  stratified  by 
hospital,  by  department,  and  by  rotation  week,  and 
selections  were  carried  out  using  a  crossed, two-stage  simple 
random  sampling  method.  The  sample  was  equally  allocated  to 
each  stratum,  and  two  levels  of  weights  were  applied  to 
balance  out  stratum  population  disproportions  as  well  as 
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variations  in  response  rates  amongst  strata. 

Student-weeks  for  electives  were  selected  by  a 
systematic  random  sampling  technique,  and  after  stratifying 
with  respect  to  calendar- weeks.  Weights  were  also  applied  to 
account  for  the  selection  ratio  and  the  non-response  within 
each  stratum. 

In  addition,  the  concept  and  possible  use  of  standard 
error  was  reviewed  in  this  chapter. 
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Chapter  4 

ANALYSIS  AND  RESULTS 


This  chapter  reports  the  study  procedures  and  results 
obtained  according  to  the  specifications,  assumptions  and 
constraints  outlined  in  Chapter  3. 

4, 1  Data  Collection 

For  each  selected  student-week,  a  time  sheet, 
accompanied  by  a  cover  letter  signed  by  the  Dean  of  Medicine 
(see  Appendix  B) ,  was  forwarded  to  the  student  one  week  in 
advance  of  his/her  designated  reporting  interval.  This 
procedure  was  followed  to  ensure  that  the  student  remembered 
to  complete  the  instrument  in  the  following  week's  period. 

The  data  collected  from  the  time-sheets  underwent 
several  phases  of  editing  and  tabulation  before  any 
meaningful  information  was  extracted.  The  first  step  was  the 
coding  and  the  keypunching  of  the  responses  onto  a  machine- 
readable  medium;  namely,  the  computer  punched-card.  Each 
card  contained  a  record  of  a  single  contact  activity  as 
perceived  by  an  individual  Phase  III  student,  and  was 
composed  of  fields  containing  data  representing  the 
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following  twelve  attributes: 

(1)  Designation  (Rotation  or  Elective) 

(2)  Student  Identification  Number 

(3)  Card  Identification  Number 

(4)  Day  of  the  Week 

(5)  Contact  Activity  Identification 

(6)  Number  of  Preceptors  Present 

(7)  Number  of  Residents  Present 

(8)  Number  of  Rotating  Interns  Present 

(9)  Number  of  Phase  III  Students  Present 

(10)  Hours  of  Contact 

(11)  Minutes  of  Contact 

(12)  Estimate  of  Instructional  Component  expressed 
as  a  Percentage 


Out  of  817  selected  student- weeks  and  417  subsequent 
responses,  a  total  of  378  [46%]  useable  returns  were 
obtained,  producing  5313  useable  activity-records.  This  gave 
an  average  of  14  activities  reported  for  every  student-week. 
The  activity  records  were  edited  in  order  to  identify  gross 
coding  and  keypunching  errors. 

As  noted  earlier,  because  of  a  rather  substantial  non¬ 
response  rate,  it  was  necessary  to  aggregate  neighboring 
strata  in  order  to  eliminate  those  with  no  (or  little) 
response.  This  procedure  was  necessary  to  enable  computation 
of  stratum  weights.  Consequently,  rotation  weeks  were 
regrouped  into  two  classifications:  "first-half"  (Rotation 
weeks  1  to  4  [ 1  to  6  for  Surgery]),  and  "second-half" 
(Rotation  weeks  5  to  8  [6  to  12  for  Surgery]).  Similarly, 
elective  weeks  were  regrouped  into  thirteen  four-week 
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Contact  time  was  then  converted  to  units  of  hours,  and 
represented  in  decimal  notation.  Preceptor-hours,  resident- 
hours,  and  intern-hours  were  calculated  for  each  activity  by 
multiplying  the  contact  time  by  the  number  of  preceptors, 
residents,  and/or  interns  respectively,  and  dividing  the 
product  by  the  number  of  Phase  III  students  in  attendance. 
This  accounted  for  the  share  of  students  recording  the 
activities.  Instructional  contact  time  was  derived  from  the 
product  of  contact  time  and  the  percent  instructional 
component. 

Student-hours  resulted  from  the  multiplication  of 
contact  time  by  the  number  of  Phase  III  students  attending 
the  activity.  This  measure  was  later  converted  to  student- 
hours  per  student-week  for  the  purpose  of  more  meaningful 
comparison  to  measures  obtained  for  other  faculties  of  the 
University  of  Alberta  (Institutional  Research  and  Planning, 
1970)  . 

All  measurements  were  weighted  according  to  the 
aforementioned  specifications.  Conseguently ,  the  resulting 
values  represented  the  unbiased  estimates  over  all  Phase  III 
student-weeks  during  the  study  year. 

With  this  framework  in  mind,  two  modes  of  analysis  were 
employed.  In  the  first  case,  faculty-hours  were  examined 
from  the  faculty  utilization  standpoint.  The  "faculty"  was 
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limited  to  preceptors,  residents  and  rotating  interns,  and 
Rotation-hours  were  analyzed  with  respect  to  hospital, 
department,  and  contact  activity.  Elective— hours  were 
tabulated  only  by  contact  activity,  as  information  on 
hospitals  and  departments  for  electives  was  not  obtained. 

The  second  approach,  that  of  student-hours  per  student- 
week,  was  derived  from  the  standpoint  of  the  time  spent  by 
the  Phase  III  students  with  various  members  of  the  teaching 
staff.  Student-hours  were  analyzed  in  a  similar  manner  to 
that  of  Faculty-hours. 

4. 2  Faculty  Contact-Time 

The  contact  that  members  of  the  Faculty  of  Medicine, 
namely  preceptors,  residents,  and  rotating  interns,  had  with 
the  Phase  III  students  during  rotation  and  elective 
activities,  was  expressed  in  units  of  Preceptor-hours, 
Resident-hours,  and  Intern-hours  respectively.  All 
measurements  were  estimated  with  respect  to  a  one-year  time 
period,  or  in  terms  of  a  one-year  operation  of  the  Phase  III 
program. 

4. 2. 1  Preceptor  Contact  Time 

Each  rotation  or  elective  activity  had  at  least  one 
preceptor  responsible  for  the  student  interns,  although  not 
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necessarily  present  at  every  activity.  A  Preceptor-hour  was 
defined  as  one  hour  of  contact  between  a  single  preceptor 
and  the  Phase  III  student  (s)  participating  on  his/her 
service.  For  example,  two  preceptors  directly  supervising 
one  Phase  III  student  for  one  hour  produced  two  Preceptor- 
hours,  while  one  preceptor  supervising  three  Phase  III 
students  for  one  hour  was  regarded  only  as  one  Preceptor- 
hour. 


Table  IV  (Appendix  A)  shows  a  breakdown  of  Preceptors- 
hours  with  respect  to  the  hospital  in  which  rotations  were 
scheduled.  Electives  were  grouped  together  and  treated  as 
though  they  were  conducted  at  a  single  location. 

The  third  column  of  Table  IV  represents  the  percentage 
of  Preceptor-hours  spent  at  the  six  hospitals  under  study, 
as  well  as  the  percentage  spent  in  rotations  and  electives. 

"Instructional- hours"  (Column  4)  were  estimated  by 
applying  the  "Percent  Instructional  Component"  (re¬ 
calculated  as  "Proportion  of  Instruction")  to  the  "Contact 
Hours"  figures.  The  reader  must  keep  in  mind  that  this  is 
not  an  objective  measure  of  direct  time,  but  a  subjective 
estimation  as  perceived  by  the  students.  Obviously,  some 
students  may  overestimate  and  some  may  underestimate,  but 
the  measurement  errors  are  assumed  to  be  of  a  cancelling 
nature,  and  in  the  process  of  aggregation,  it  was  expected 
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that  the  estimation  provided  an  unbiased  estimator. * 

To  illustrate  how  to  interpret  Table  IV,  consider  the 
case  of  the  University  Hospital.  The  estimated  amount  of 
instructional  preceptor  contact  with  Phase  III  students 
during  rotations  at  this  hospital  was  15,077  hours  over  a 
period  of  one  year.  Dividing  by  52  weeks,  this  can  be 
expressed  as  290  Preceptor-hours  per  week.  If  we  assume  that 
there  exist,  say,  twenty  preceptors  supervising  Phase  III 
students  at  the  University  Hospital  at  any  given  time,  the 
average  contact  time  then  becomes  14.5  hours  per  preceptor 
per  week. 

This  may  be  illustrated  in  yet  another  manner.  Assuming 
that,  in  any  given  faculty,  a  professor  lectures  and 
consults  with  his  students  for  eighteen  hours  each  week, 
over  forty-eight  weeks  per  year,  and  in  applying  these 


1  The -estimation  of  instructional  component  was 
examined  in  a  separate  study  entitled  "Scaling  Techniques 
and  Subjective  Evaluation:  Two  Measures  Estimating  the 
Instructional  Utility  of  the  Phase  III  Program  of  Medical 
Education",  by  Nusbaum,  M.  [unpublished  paper  for  course 
HSA531,  University  of  Alberta,  May  1976].  The  study  can  be 
found  in  its  entirety  in  Appendix  C. 


The  study  indicated  that,  inferentially ,  there  was  a 
degree  of  reliability  between  the  responses  using  the  two 
independent  measures,  and  that  Phase  III  students  who 
reacted  positively  to  the  instruments  were,  at  the  least, 
consistant  in  their  responses,  [p.26] 
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figures  to  preceptor  contact  at  the  Edmonton  General 
Hospital*  it  can  be  shown  that  the  instruction  provided  by 
the  hospital  is  equivalent  to  about  nine  full-time  faculty- 
members  [7*732/(18  X  48)  =  8.9  ]. 

The  reader  should  note  that  Preceptor-hours  for 
electives  make  up  approximately  43%  of  the  total*  although 
less  than  [3024  /  (3024  +  5534)  ]  =  36%  of  the  total  number 
of  student-weeks  were  originally  assigned  for  electives. 

This  is  probably  attributed  to  the  higher  attention  paid  by 
preceptors  to  the  students  taking  electives  as  a  result  of 
the  more  individualized  program. 

Table  V  similarly  shows  a  breakdown  of  Preceptor-hours 
by  departments.  Note  that  only  rotations  are  included  in 
this  table  due  to  the  previously-mentioned  impracticality  of 
collecting  departmental  information  for  electives.  In 
addition*  values  are  tabulated  over  all  hospitals;  further 
breakdown  was  aborted  for  fear  of  losing  data  reliability 
due  to  insufficient  sample  size. 

Preceptor-hours  for  both  rotations  and  electives  are 
tabulated  by  Contact  Activity  in  Table  VI. 

Figures  3  and  4  diagramatically  reiterate  the 
information  found  in  Tables  IV*  V*  and  VI. 
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FIGURE  3 


PRECEPTOR-HOURS  BY  DEPT . 


PRECEPTOR-HOURS  BY  HOSPITRL 
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PRECEPTOR-HOURS  BY  ACT  (ROT) 


7  CALL 


CLIN1 


, 


80 


4. 2. 2  Resident  Contact  Time  and  Rotating  Intern  Contact  Time 

A  Resident-hour  was  defined  similarly  to  that  of  a 
Preceptor- hour :  one  hour  of  contact  between  a  single 
resident  and  the  Phase  III  student  (s)  participating  on 
his/her  service. 

Tables  VII,  VIII,  and  IX  display  Resident-Contact 
broken  down  by  hospital,  department,  and  by  contact  activity 
respectively.  Once  again,  this  data  is  available 
diagramatically  in  Figures  5  and  6. 

An  Intern-hour,  or  one  hour  of  contact  between  a  single 
rotating  intern  and  the  Phase  III  student (s)  participating 
under  his  supervision,  is  the  measurement  unit  used  in 
Tables  X,  XI,  and  XII. 

Figures  7  and  8  diagramatically  present  the  tabular 


data. 
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FIGURE  5 

RESIDENT-HOURS  BY  DEPT  . 


RESIDENT-HOURS  BY  HGSPITRL 


FIGURE  6 
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RESIDENT-HOURS  BY  RCT  (ELECj 


RESIDENT-HOURS  BY  RCT  (ROT) 
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FIGURE  7 

INTERN-HOURS  BY  DEPT. 


INTERN-HOURS  BY  HOSPITHL 
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FIGURE  8 
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INTERN-HOURS  BY  RCT  (ELEC) 


INTERN-HOURS  BY  ROT  (ROT) 
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4*3  Student  Contact  Time 

The  three  previous  sections  of  this  chapter  described 
faculty-student  contact  from  the  standpoint  of  the  faculty- 
members  [ ie.  an  estimate  of  the  time  actually  spent  by  the 
faculty  with  the  students].  This  section  outlines  the 
estimation  of  student  contact  time  with  faculty  members. 

Student  contact  was  defined  in  terms  of  student-hours, 
and  in  order  to  be  compatible  with  the  units  chosen  by 
Institutional  Research  and  Planning,  this  unit  was  further 
reduced  to  "Student- hours  per  student-week".  As  before,  all 
measurements  were  tabulated  with  respect  to  a  one-year  time 
period. 


A  Student-hour  was  defined  as  one  hour  of  contact 
between  a  single  Phase  III  student  and  a  member  [or  members] 
of  the  Faculty  of  Medicine  [specifically  preceptors, 
residents  and/or  rotating  interns].  As  before,  if  a  student 
met  for  one  hour  with  two  faculty-members,  this  constituted 
only  one  Student-hour.  On  the  other  hand,  two  students  in 
contact  with  one  faculty— member  for  one  hour  described  two 
Student-hours. 


Table  XIII  displays  student  contact  broken  down  by  the 
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hospital  in  which  rotations  were  scheduled.  Once  again, 
electives  were  tabulated  only  in  total.  Column  one  lists  the 
participating  hospitals,  and  column  two  shows  the  total 
number  of  Student-hours  logged  at  each  hospital  over  a  one- 
year  period.  For  example,  at  the  University  of  Alberta 
Hospital,  there  was  a  total  of  60,814  student-hours 
generated  by  Phase  III  students. 

Column  three  of  Table  XIII  lists  the  percentage  of 
total  Student-hours  spent  at  each  hospital,  and  the  fourth 
column  tabulates  the  estimated  number  of  student-weeks  spent 
at  each  hospital. 

The  fifth  column  of  Table  XIII  displays  the  number  of 
hours  of  contact  per  average  student-week,  followed  by 
instructional  Student- hours.  Instructional-hours  per 
student-week,  and  the  proportion  of  instruction  in  columns 
six,  seven  and  eight  respectively.  As  before, 

"instructional”  time  is  a  function  of  the  students' 
estimates  of  "Percent  Instruction". 

A  similar  tabulation  by  department  follows  in  Table 
XIV,  as  well  as  tabulations  by  contact  activity  for 
rotations  (Table  XV)  and  for  electives  (Table  XVI).  Graphic 
displays  of  the  above  can  be  found  in  Figures  9  and  10. 
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FIGURE  9 

STUDENT-HOURS  BY  DEPT  ° 


STUDENT-HOURS  BY  HOSPITHL 
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FIGURE  10 

STUDENT-HOURS  BY  RCT  (ELEC) 


STUDENT-HOURS  BY  RCT  (ROT) 
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Interpretation  can  be  illustrated  in  a  manner  similar 
to  that  of  Faculty-hours.  As  an  example,  consider  the  10,288 
instructional  Student— hours  spent  on  Obstetric  Rotations 
during  the  year  [see  Table  XIV].  Dividing  by  fifty-two 
weeks,  it  can  be  shown  that  198  Student— hours  were  spent  in 
that  department  every  week.  Assuming  that  there  are,  say, 

200  Phase  III  students  enrolled  in  the  Faculty,  it  can  be 
said  that  an  av erage  of  1.0  hours  per  week  are  spent  in 
Obstetrics  by  each  Phase  III  student  with  direct  contact 
with  one  or  more  staff. 

One  may  observe  from  Tables  XIII  and  XIV  that  each 
student  spent  about  thirteen  hours  per  week  in  direct 
contact  with  one  or  more  staff  members,  and  this  is  more  or 
less  the  same  for  all  hospitals,  and  electives.  The  minimum 
is  11.7  hours  per  week  at  the  Edmonton  General  Hospital,  and 
the  maximum  is  13.6  hours  per  week  at  the  University 
Hospital.  The  variation  amongst  hospitals  is  probably  due  to 
the  difference  amongst  departments.  There  are  substantial 
variations  amongst  departments,  ranging  from  a  maximum  of 
14.6  hours  per  week  for  Surgery,  to  a  minimum  of  9.1  hours 
per  week  for  Psychiatry. 

4. 4  Summary  of  Findings 

The  following  is  a  synopsis  of  the  findings  of  this 


study. 
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4.4.1  Preceptor^Hours 

The  total  contact  between  Phase  III  students  and 
preceptors  of  the  Faculty  of  Medicine,  from  September  4, 

1975  to  September  2,  1976,  was  136,548  Preceptor-hours.  Of 
this  total,  57%  of  the  contact  occured  during  rotations  and 
43%  during  electives.  The  number  of  instructional  preceptor 
contact-hours  totalled  73,228,  denoting  an  instructional 
factor  of  53.6%. 

Rotations  in  Surgery  obtained  the  highest  proportion  of 
preceptor  contact,  reaching  43%.  Conversely,  the  department 
yielding  the  greatest  proportion  of  preceptor-instruction 
was  Medicine  I  [58.0%],  and  the  lowest  proportion  of 
preceptor- instruction  occurred  in  Surgery  [47.0%]. 

With  respect  to  the  distribution  of  Preceptor-hours 
over  hospitals,  the  University  of  Alberta  Hospital  claimed 
the  most  [37%]  of  all  Preceptor-hours,  and  the  Alberta 
Hospital  (Oliver)  claimed  the  least  [2%].  Ward  Rounds  [31%] 
and  Operating  Room  [33%]  were  the  activities  providing  the 
bulk  of  preceptor— contact  for  rotations,  and  the  same  held 
true  for  elective  activities  [24%  and  33%,  respectively]. 

As  one  might  expect,  the  greatest  proportion  of 
instruction  from  preceptors  occured  at  Seminars  [86.7%],  and 
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the  least  daring  Operating  Room  sessions  [  4  1.256  ]. 

4. 4. 2  Student-Hours 

The  total  number  of  hours  of  contact  by  the  Phase  III 
students  with  faculty  over  the  study  year  totalled  196r818r 
of  which  55.8%  was  perceived  by  the  students  as  being 
instructional.  Instructional  contact  was  highest  at  the 
University  of  Alberta  Hospital  [13.6  Student-hours  per 
student-week],  and  lowest  at  the  Edmonton  General  Hospital 
[11.7  Student-hours  per  student-week]. 

It  is  interesting  to  note  that  the  overall  proportion 
of  instruction  was  relatively  homogenous  over  hospitals,  as 
well  as  over  rotations  [ 55%  ]  and  electives  [57%]. 

On  the  average,  contact  by  the  students  with  faculty 
was  23.0  hours  per  student-week,  and  out  of  this,  12.8  hours 
[55.8%]  were  considered  instructional.  The  latter  figure  is 
equivalent  to,  say,  four  courses  per  week  at  three  hours  per 
course. 

Surgery  rotations  demanded  the  greatest  student  contact 
with  faculty  [29.3  hours  per  week],  but  the  corresponding 
instructional  component  [49.6%]  was  the  lowest  of  all 
departments,  yielding  only  14.7  hours  of  instruction  per 
student-week.  The  fewest  number  of  hours  spent  by  students 
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occurred  in  Psychiatric  rotations  [  16.7  hours  per  week],  of 
which  9.1  hours  were  considered  of  instructional  value. 

4. 5  Estimation  of  Standard  Error 

In  order  to  evaluate  the  reliability  of  various 
estimates,  and  to  obtain  appropriate  statistical 
interpretation,  standard  errors  were  computed  for  the 
following  estimates: 

(1)  Total  Preceptor- hours 

(2)  Total  Resident-hours 

(3)  Total  Student-hours 

(4)  Total  Instructional  Preceptor- hours 

(5)  Total  Instructional  Resident-hours 

(6)  Total  Instructional  Student-hours 

The  standard  error  formula  discussed  in  Chapter  3  is 
based  on  a  simple  random  sample  (SRS)  without  replacement 
within  each  stratum,  as  previously  defined.  However,  the 
sampling  plan  of  the  present  study  was  not  an  SRS,  but  a 
modified  cluster  sampling  technique.  In  spite  of  this,  and 
in  spite  of  the  high  degree  of  non-response  encountered  in 
the  study,  the  formula  was  used  to  derive  an  approximation 
of  tlie  standard  error.  It  should  be  noted  that  the  standard 
error  may  be  somewhat  underestimated,  due  to  the  nature  of 
the  clustering  technique  and  the  problems  of  non-response. 

Tabulations  of  standard  error  and  related  calculations 
can  be  observed  in  Table  III.  To  further  illustrate  the 
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Table  III 

Estimation  of  Standard  Error 
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(1) 


(2) 


(3) 


(4) 


Measure 


Total 


Enh(l -fh) s 


2 

h 


Relative 


S.  E. 

(Z) 


•{ — - 


Preceptor- hours 

136,548 

1 

1 

1 1,566,616 

« 

1 

5.  74 

Resident- hours 

94,202 

1 

7,769,251 

1 

7.  27 

Student-hours 

196,818 

1 

7,916,590 

1 

2.83 

Inst.  Prec-hrs 

73,228 

i 

8,135,654 

1 

8.37 

Inst.  Res^-hrs 

46,060 

I 

2,286,014 

1 

8.  82 

Inst.  Stud-hrs 

109,883 

j _ 

1 

1 

_ i _ 

6,079, 559 

I 

1 

— i — 

4.  33 

method  by  which  standard  error  was  calculated,  the  following 
steps  were  used  in  the  case  of  Total  Preceptor-hours. 


As  previously  identified  in  this  chapter,  the  total 
number  of  Preceptor— hours  as  estimated  from  the  survey 
results  was  136,548.  Using  stratification  by  hospitals, 
departments,  rotation  weeks,  and  elective  weeks  over  all 
st udent-weeks ,  the  sample  variance  resulted  in  61,40~>,308 
hours  squared.  The  square  root,  or  sianciard  error  vuas  found 

to  be  7836.1  hours. 


A  more  meaningful  expression  of  standard  error  is  that 
of  relative  standard  error,  or  the  coefficient  of  variation. 
This  is  simply  the  standard  error  expressed  as  a  percentage 


•  v 


. 

94 


of  the  total  sample  estimate.  Referring  once  again  to  the 
example,  the  relative  standard  error  for  Total  Preceptor- 
hours  was  estimated  at  5.74%  [  7,836.1/136,548  x  100%].  Of 
the  six  estimates  tested  for  standard  error  (see  Table  III) , 
the  most  consistent  was  that  of  Total  Student-hours  (2.83% 
relative  S.E.).  This  was  expected,  mainly  because  of  the 
large  sample  size  (8,558  student  weeks)  yielding  the  largest 
estimate  of  total  contact  time  (196,818  hours). 

Also  expected  was  the  high  relative  standard  error  for 
Instructional  Resident-hours.  In  this  case,  the  estimate  of 
contact  time  was  small  (46,060  hours),  and  the  variation 
amongst  the  students*  estimates  of  "Percent  Instructional 
Component"  was  large. 

4. 6  Opinion  Survey  of  Phase  III  Students 

As  a  follow-up  procedure  to  the  Phase  III  Faculty- 
Student  Contact  Time  survey,  the  author  distributed  an 
additional  questionnaire  to  all  Phase  III  students 
previously  surveyed  and  still  remaining  in  the  Faculty  of 
Medicine  as  at  September,  1976.  A  sample  of  this 
questionnaire  can  be  found  in  Appendix  D. 

A  total  of  110  students  received  the  follow-up 
questionnaire,  and  out  of  these  52  returned  them  completed. 
This  gave  a  rather  disappointing  response  rate  of  47%. 
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The  students  were  asked  general  questions  about  the 
survey  and  about  the  Phase  III  program.  Most  respondents 
reacted  favourably  to  the  survey,  and  found  it  to  be 
potentially  valuable  in  monitoring  the  level  of  faculty- 
student  contact.  Only  about  10%  of  the  respondents  gave 
negative  comments,  and  these  were  mainly  related  to  problems 
such  as  too  little  time  to  fill  in  survey  time-sheets.  It 
must  be  understood  that  the  opinion  survey  respondents  were 
more  than  likely  those  who  originally  responded  to  the 
primary  contact-time  survey,  and  as  a  result,  negative 
comments  regarding  the  survey  are  probably  underestimated. 

Most  of  the  52  responding  students  indicated  that  they 
had  been  asked  to  complete  between  2  and  4  time  sheets, 
although  some  reported  that  they  had  completed  as  many  as  6 
or  7.  Only  2  of  the  respondents  claimed  that  they  had  never 
received  a  time  sheet  during  the  course  of  the  study  year. 
Most  of  those  who  didn't  return  all  time  sheets  to  the 
investigator  (48%)  indicated  that  they  simply  "forgot”  to  do 
so  (13%) ,  or  they  were  just  not  sufficiently  interested  in 

the  study  (6%). 

The  majority  of  the  respondents  (85%)  ,  as  indicated  by 
the  opinion  survey,  felt  that  the  faculty-student  study  was 
not  too  great  an  imposition  on  their  time.  Once  again,  85% 
of  the  respondents  clearly  understood  the  purpose  and  need 
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for  the  study ,  although  some  would  have  preferred  a  somewhat 
clearer  explanation  via  more  preliminary  written  material. 

Only  65%  of  the  respondents  felt  that  the  study 
actually  reflected  their  student  intern  activity,  and  40% 
suggested  that  additional  information  should  have  been 
collected  by  the  investigator.  Suggestions  included 
estimates  of  "clinical  teaching  time",  as  well  as  time  spent 
in  "technical  sessions". 

A  majority  of  the  students  (65%)  rejected  the  validity 
of  "Percent  Instructional  Component"  as  a  measure  of  faculty 
contact.  Interestingly  enough,  the  two  measures  obtained  for 
instructional  utility,  as  reported  in  the  paper  found  in 
Appendix  C,  demonstrated  at  the  very  least,  a  certain 
consistency,  or  reliability,  in  the  students'  estimates. 

Most  respondents  felt  that  a  mail  strike,  which 
occurred  in  December  1975,  did  not  hinder  their  reporting 
effort.  However,  survey  response  rate  dropped  from  65%  to 
20%  during  that  period.  This  inconsistancy  can  probably  be 
attributed  to  the  fact  that  respondents  on  the  follow-up 
questionnaire  more  than  likely  represented  those  Phase  III 
students  who  responded  to  the  initial  questionnaire. 

In  general,  the  respondents  suggested  that  subsequent 
studies  investigating  faculty— student  contact  time  be 
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carried  out  on  a  regular  basis.  They  were  encouraged  by  the 
effort  demonstrated  by  the  Office  of  the  Dean  of  Medicine, 
and  felt  that  more  effective  staff-student  interaction  may 
be  related  to  the  amount  of  administrative  interest. 
However,  a  number  of  students  suggested  a  parallel 
questionnaire  format  investigating  the  quality  of  staff- 
student  contact.  They  reasoned  that  effective  decision¬ 
making  by  the  administration  could  only  be  accomplished 
using  a  more  comprehensive  data  base. 

Although  the  comments  obtained  through  the  opinion 
survey  may  or  may  not  be  representative  of  the  student 
population,  the  information  can  possibly  be  useful  to 
researchers  attempting  further  study  into  faculty-student 
contact. 

4.7  Follow-Dp  Faculty  Survey 


Six  faculty  members  participating  in  the  Phase  III 
program  were  interviewed  by  the  author.  An  example  of  a 
typical  interview  format  is  found  in  Appendix  E. 

Four  of  the  faculty  members  knew  nothing  about  the 
Phase  III  Faculty-Student  Contact  Time  study,  but  those  who 
did  generally  supported  it.  The  faculty  members  were  asked 
to  validate  some  of  the  information  obtained  in  the  study  by 
commenting  on  the  curricular  framework  used  in  their 
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particular  rotation  (s)  or  elective (s) .  Comments  were 
obtained  on  the  number  of  staff  (resources)  available  in 
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each  Rotation  or  Elective,  as  well  as  comments  pertaining  to 
the  typical  daily  activities  of  a  Phase  III  student  under 
the  faculty  member's  direction. 

The  author  found  no  consistency  amongst  the  faculty 
members'  responses  in  terms  of  rotation  or  elective  formats. 
It  was,  however,  generally  expressed  by  some  faculty  members 
that  staff  resources  were  presently  overloaded,  and  that 
more  effort  should  be  spent  allocating  these  resources  in  an 
effective  manner. 

The  interviews  did  not  yield  any  significant 
conclusions,  other  than  a  very  general  validation  of  the 
study  results.  The  author  argues  that  such  an  analysis  is 
only  effective  when  designed  and  organized  under  stringent 
methodological  frameworks.  Unfortunately,  the  faculty 
interviews  did  not  meet  this  goal. 

The  question  of  validity  of  this  study  can  be 
summarized  in  the  following  manner.  The  "independent  study" 
[found  in  Appendix  C]  signified  a  certain  degree  of 
reliability  amongst  the  students'  assessment  of 
"instructional  component",  although  no  such  relationship  was 
found  with  faculty  members  who  commented  positively  on  the 
overloading  of  staff  resources.  However,  there  may  be 
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evidence  of  a  possible  correlation  between  the  student 
respondents  and  faculty  members,  and  further  study  may 
indeed  uncover  this  fact. 

4- 8  Summary 

The  data  obtained  from  the  time-sheets  over  the  one- 
year  study  period  was  compiled,  edited,  coded,  and  analyzed 
under  rigorous  methodological  scrutiny.  Results  were 
tabulated  by  rotation  and  elective  activities,  as  well  as  by 
hospital,  department,  and  activity  type.  Estimation  of 
contact  time  was  organized  from  the  students1  standpoint  (in 
terms  of  Student- hours) ,  and  from  the  faculty's  standpoint 
(by  examining  Preceptor-,  Resident-,  and  Rotating  Intern- 
hours)  . 

From  the  above  estimates,  relative  standard  error  (or 
coefficient  of  variation)  estimations  were  derived,  ranging 
from  2%  to  9%.  These  results  appeared  reasonably 
satisfactory  to  the  author,  and  enabled  him  to  make  several 
inferences  to  the  general  population  that  could  prove  useful 
to  future  manpower  planning. 

Two  follow-up  opinion  surveys  were  conducted,  one  with 
the  Phase  III  students  and  the  other  with  selected 
participating  physicians.  The  former  survey  was  most  useful 
in  obtaining  impressions  of  the  study  from  the  students' 
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point  of  view.  However ,  the  physician  interviews  lacked  the 
organization  and  forsight  to  be  of  much  use  in  evaluating 
the  Phase  III  Faculty-Student  Contact  Time  study. 
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Chapter  5 

SUMMARY  AND  RECOMMENDATIONS 


The  Phase  III  Faculty-Student  Contact  Time  Study 
involved  some  200  students,  60  faculty  members,  and  3 
research  personnel,  over  a  time  period  of  14  months. 
Approximately  5000  activity  records  were  collected,  edited, 
and  stored,  and  a  number  of  special  computer  programs  were 
written  and  used  to  analyze  the  data.  Yet,  despite  this 
extensive  resource  utilization,  the  question  arose  whether 
or  not  one  can  conclude  any  significant  facts  from  this 
study.  To  fairly  asses  this  question,  one  must  examine  the 
initial  study  objectives,  and  relate  them  to  the  study 
findings. 

5. 1  Summary  and  Evaluation 

The  primary  objective,  as  stated  in  Chapter  1,  was  to 
estimate  the  actual  amount  of  faculty— student  contact  time 
over  both  rotation  and  elective  activities.  This 
investigation  provided  estimates  through  the  use  of 
carefully— defined  sampling  procedures,  and  by  adjusting  the 
sample  results  with  appropriate  weights.  The  Phase  III 
students  were  asked  to  respond  over  a  time  period  of  52 
calendar  weeks,  and  this  enabled  data  to  be  collected 
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reflecting  the  students*  perceptions  of  contact  time 
throughout  one  year  of  the  academic  cycle.  Using  different 
sampling  strategies,  both  rotation  and  elective  activities 
were  investigated,  and  the  results  were  adjusted  to  allow 
direct  comparisons.  In  addition,  contact  time  was  examined 
in  two  perspectives:  (1)  from  the  faculty's  point  of  view, 
in  terms  of  the  amount  of  contact  they  experienced  with  the 
students,  and  (2)  from  the  students'  point  of  view, 
identifying  their  contact  with  the  faculty. 

The  second  objective  was  to  compare  contact-time  over 
participating  hospitals  and  departments.  In  this  respect, 
the  population  of  student- weeks  for  rotations  was  stratified 
into  25  unique  hospital-department  groups,  and  samples  were 
obtained  from  each.  The  author  analyzed  data  within  and 
amongst  these  strata  in  terms  of  preceptor,  resident, 
intern,  and  student  contact- time.  Results  showed  a 
significant  degree  of  variation  among  the  groups. 

The  identification  and  comparison  of  direct  instruction 
amongst  hospital-department  groups  was  the  third  objective. 
This  aspect  was  analyzed  in  two  ways.  In  the  first  case,  the 
students  were  asked  to  subjectively  estimate  a  percentage  of 
"instructional  component"  for  each  identified  period  of 
contact.  This  method  produced  an  adjustment  factor  which, 
when  applied  against  the  raw  data,  provided  the  investigator 
with  an  estimate  of  direct  instructional  time.  The  second 
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method,  used  in  a  separate  study,  instructed  the  students  to 
rank  each  Phase  III  activity  in  the  order  of  "greatest 
instructional  value”,  and  a  subsequent  correlation  of  these 
results  with  those  of  the  first  method  yielded  a  significant 
compatability  relationship.  These  independent  measures 
allowed  the  conclusion  to  be  drawn  that,  at  the  very  least, 
the  subjective  value  of  the  students*  assessment  of 
instructional  time  was  consistant.  Over  all  strata,  the 
average  instructional  component  was  about  54%,  suggesting 
that,  as  far  as  the  students  were  concerned,  about  half  of 
all  contact  time  was  spent  in  direct  instruction. 

The  final  objective  was  to  evaluate  the  methodology 
employed  in  this  study.  The  theoretical  rationale  for 
utilizing  a  sampling  design  was  thoroughly  discussed  in 
Chapter  2,  as  was  the  rationale  for  the  choice  of 
respondents  and  the  time  frame.  The  practical  implications, 
however,  necessitate  some  further  examination. 

Given  that  the  time  frame  was  a  one- year  period,  the 
sampling  methodology  did  present  certain  potential  sources 
of  error.  The  response  rate,  although  initially  high  at  80%, 
dropped  drastically  after  about  12  calendar  weeks  (to  about 
51%),  and  never  again  regained  the  previous  level.  This  was 
about  the  time  that  the  students  began  to  receive  their 
second  and  third  time-sheets,  and  at  that  point  they  were 
less  willing  to  participate  than  at  the  onset  of  the  study. 
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In  addition,  during  a  6— week  Canadian  Postal  Workers*  strike 
in  December,  1975,  there  showed  a  marked  drop  in  response 
rate,  even  though  the  researchers  made  every  attempt  to 
compensate. 

Because  of  this  low  rate  of  response  and  the  subsequent 
presence  of  “empty  cells'*  in  some  strata,  the  replication 
scheme  originally  included  in  the  sampling  design  had  to  be 
eliminated  from  the  study  methodology.  Because  of  this,  the 
author  was  forced  to  exclude  a  technique  which  would  have 
provided  other  estimates  of  standard  error. 

Also  because  of  the  low  response  rate,  many  strata  had 
to  be  combined  in  the  analysis  phase.  This  was  not 
anticipated  in  the  initial  design  of  the  study  and,  as  in 
any  mid-stream  adjustment,  some  reliability  of  the  results 
was  lost.  Another  aspect  that  was  affected  by  stratum 
collapsing  was  the  level  of  detail  of  the  stratification 
scheme.  The  resulting  consequence  was  that  the  sample  did 
not  represent  the  population  as  well  as  it  could  have. 

A  major  weakness  in  the  sampling  design  and  the 
analysis  of  data  lies  in  the  absence  of  a  study  relating  to 
the  effects  of  time  on  the  measurements  obtained.  Although 
provision  was  made  in  the  stratification  scheme  to  account 
for  this  factor,  various  constraints  prohibited  the  author 
from  determining  whether  or  not  systematic  differences 
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existed  between,  say,  measurements  obtained  during  the 
Winter  vis  a  vis  measurements  obtained  during  the  Summer.  As 
a  result,  in  future  study  it  may  be  difficult  to  determine 
whether  a  time— frame  of  less  than  one  year  could  adequately 
provide  estimates  representing  an  annual  cycle  of  student- 
faculty  contact. 

5. 2  Recommendations 

The  following  are  the  author* s  recommendations  for 
further  investigation  into  Phase  III  Faculty  Student  Contact 
Time.  These  recommendations  are  based  solely  upon  the 
results  of  the  present  study. 

5.2.1  Recommendation  #1 


In  the  author's  opinion,  the  methodology  used  in  this 
study  is  (subject  to  the  modifications  outlined  below) 
sufficiently  adequate  to  warrant  its  use  in  further  research 
of  this  nature.  The  results  appeared  comparable  to  measures 
used  in  faculties  other  than  the  Faculty  of  Medicine,  and 
the  author  suggests  that  the  information  may  be  useful  when 
applied  directly  to  inter-faculty  comparisons.  It  is 
recommended  that  a  similar  study  be  repeated  by  the  Faculty 
of  Medicine  on  a  regular,  periodic  basis,  in  order  to 
maintain  an  up-to-date  base  of  data.  In  doing  so, 
information  would  not  only  be  valuable  for  comparing  contact 


«■  ■  . 


.  D  \  li  >1 


- 


’ 


. 


.  —  x  •  ■  actios  i 


•  • 

t?  ,  k  ni>io  *  .  n»a  al 


■ 


•  1  5  ;  d  |  .:i 


. 


. 


- 


- 


106 


time  between  faculties*  but  would  also  provide  a  historical 
data  base  with  which  Faculty  administrators  could  observe 
and  monitor  system  changes  over  time. 

5-2.2  Recommendation  #2 


Although  the  students  and  faculty  generally  supported 
the  objectives  of  this  study,  the  rate  of  response  was 
relatively  low.  It  is  recommended  that,  for  any  future 
studies*  the  possibility  of  a  time-frame  of  duration  shorter 
than  one  year  be  considered,  in  an  attempt  to  maintain  the 
interest  of  the  respondents.  As  an  example,  perhaps  a  series 
of  cross-sectional  studies,  say  four  per  calendar  year, 
would  be  more  effective  in  increasing  and  maintaining  an 
acceptable  rate  of  response.  Other  measures  might  include 
(1)  increased  enforcement  of  students*  response,  (2)  the 
balancing,  or  equalization  of  the  number  of  times  a  student 
is  called  upon  to  respond  during  a  calendar  year,  and  (3) 
maintaining  more  up-to-date  records  of  current  addresses  of 
the  students.  An  improved  rate  of  response  could  facilitate 
the  use  of  the  replication  method  for  estimating  standard 
error,  as  well  as  a  more  extensive  (and  detailed) 
stratification  scheme.  Such  a  sampling  design  should 
incorporate  equalized  responses  from  each  student. 
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5.2.3  Recommendation  #3 


The  author  recommends  that  several  other  independent 
measures  be  taken  to  ascertain  the  students'  assessment  of 
"instructional  component"  of  contact  time.  The  students,  the 
faculty,  as  well  as  the  author  agreed  that  this  facet  was  a 
major  weak  point  in  the  study.  It  was  also  pointed  out  that 
a  more  specific  definition  of  "instruction"  would  help  to 
facilitate  more  accurate  measurements  of  instructional 
contact  time. 

5.2.4  Recommendation  #4 

It  is  suggested  that  survey  instruments  other  than  mail 
questionnaires  be  evaluated  as  alternative  modes  for  future 
study.  Alternatives  could  possibly  include  the  use  of 
interviewers  or  observers  for  all  respondents  in  the  sample, 
or  the  use  of  sub*-samples  to  control  costs.  Although  the 
cost  would  increase,  this  may  help  control  the  response  rate 
and  measurement  errors,  and  hence  improve  the  quality  of  the 
results. 

5. 2. 5  Recommendation  15 


The  final,  and  perhaps  the  most  vital,  recommendation 
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is  that*  because  this  study  was  only  of  a  descriptive 
nature,  the  results  should  be  used  only  as  a  guideline  in 
the  decision-making  process.  Further  norma ti ve— type  study 
into  the  qua lit y  of  faculty-student  contact,  as  well  as  into 
Py° contact  time  under  future  conditions,  should  be 
considered  before  resource  planning  becomes  dependent  on  the 
data  resulting  from  this  type  of  research- 

5.3  Future  Research 


This  investigation  has  provided  information  concerning 
the  level  of  resource  utilization  within  the  Phase  III 
program.  Although  the  results  are  useful  [and  necessary]  in 
establishing  an  initial  data  base,  the  following  questions 
are  still  unanswered: 

(1)  What  is  the  value  of  the  present  contact 

between  staff  and  students? 

(2)  Can  the  effectiveness  of  the  contact  be 

improved? 

(3)  How  can  system  slack,  or  non-instructional 

contact-time,  be  evaluated? 

Only  by  answering  these  questions  can  decision-makers 
institute  effective  change  and  improve  the  calibre  of  the 
trained  medical  professional. 
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Table  IV 

Annual  Preceptor-Contact  by  Hospital 
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Faculty  1 
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|  4,736 

0-  445 
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0.532 

Alta-  Hosp.  | 

1,225  | 
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1 

0.  508 
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77,794  1 
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|  39,267 
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Table  V 

Annual  Preceptor-Contact  by  Department 
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Table  VII 

Annual  Resident-Contact  by  Hospital 


« - n - - t 

I  I  | 

I  (1)  I  (2)  | 

|  HOSPITAL  |  Total  | 

I  |  Faculty  | 

I  |  Hours  J 

I  I  I 

J - 1 - 1 

I  1  I 
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|  General  |  3,288  j 

|  Misericordia  J  4,532  | 

|  Camsell  |  1,378  J 

|  Alta.  Hosp.  |  29  | 

I  I  I 
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Table  VIII 

Annual  Resident-Contact  by  Department 
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FACULTY  OF  MEDICINE 
oi  t  (Cii  of  Tiif-  r>:  an 


THE  UNIVERSITY  OF  ALHERTA 
EDMONTON.  CANADA 
TliG  2G3 


Phase  III  Medical  Students 


Rej_  faciil  ty/Studcnt  Contact  Time:  A  Survey  of  the  Phase  III  M.D.  Program 

You.  have  been  selected  as  a  respondent  for  the  week  specified  on  -the 
TIME-SHEETS  attached.  Please  fill  out  the  TIME-SHEETS  as  accurately  as 
possi ble. 

Should  any  difficulties  arise,  please  contact  Mr.  Michael  Nusbeum  (432  5817 
office;  439  7989  home),  Research  Assistant  for  the  project,  or  Dr.  K.S.  Bay 
(432  0535)  of  the  Division  of  Health  Services  Administration,  Faculty  of 
Medicine,  Rm.  13-104  C.S.B. 


In  order  to  facilitate  accurate  study  results,  it  is  imperative  that  each 
TIME-SHEET  be  completed  and  mailed  in  as  soon  as  possible.  Failure  to  do  so 
will  most  definitely  invalidate  the  study,  and  a  great  deal  of  time  and  money 
w ill  ha ve  been  wa s t e d . 


For  your  convenience,  a  return  address  label  is  attached  to  the  back  of  the 
TIME-SHEETS.  Please. fold,  staple,  and  mail  the  SHEETS  promptly  upon 
completion  at  the  end  of  the  week's  period. 

Thank  you  for  your  participation  in  this  research  project. 


1).  F-  Camer.on  f  M.  LI. 

Dean,  Faculty  of  Medicine 
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ft! AM.  Ill  M.l).  PROGRAM 
Tin'  University  of  Alberto 

FACUI  TY- -STUDENT  CONTACT  _T  IMF  SURVf  V 
Seven  -Doy  flee,  or  cl  of  Student  Contact  1  i  me 


Purpose 


The  purpose  of  this  survey  is  to  estimate  the  total  amount  of  direct 
FACULT’r  -  S  t  OPEN 1  CONTACT  T 1  Mr  during  Phase  I  I  I  lor  the  1 975/7h  academic  year. 

Definition 


FACULTY-STUDE !1T  CONTACT  TIME  is  the  time  taken  in  formal  and  informal 
sessions  during  which  Phase  111  students  are  observing  or  receiving  instruc¬ 
tions  under  direct  supervision  of  one  or  more  preceptors  and/or  other  radical 
staff  members,  including  residents  and  interns.  "Direct  supervision"  means 
that  the  student  is  in  the  physical  presence,  or  within  sight,  of  the  super¬ 
visor  . 


Instruct i ons 

You  have  been  provided  a  set  of  TIME-SHEETS  (or  survey  forms)  on 
which  you  are  to  maintain  a  simple  chronological  record  of  your  FACULTY- 
STUDENT  CONTACT  TIME  during  the  seven-day  period,  starting  at  3  a.m.  on 
Thursday  of  the  week,  We  suggest  that  you  record  every  day  rather  than  try 
to  fill  them  out  at  the  end  of  the  week. 

Please  complete  the  TIME-SHEET  according  to  the  instructions  below: 

Col.  (1) 


Col.  (2) 


Cols.  (3)-(6)  In  Columns  (3)  through  (6) ,  record  the  average  number  of  pre¬ 
ceptors,  residents,  and  interns  who  v/ere  present  during  the 
con  tact  act i vi ty .  Pe  sure  to  include  yourself  when  you  record 
the  number  of  Phase  111  students  present.  Finally,  guest 
lecturers,  etc.,  arc  considered  "preceptors"  for  the  purposes 
of  this  study. 

to  the  nearest 
the  final  zero 
consumes  i>0 


[ol,  (7)  fill  in  the  box  wi th  the  length  of  time,  rounded 

10  minutes,  convulsed  by  the  activity.  Note  that 
has  been  filled  in  for  you;  thus,  if  an  activity 
minutes,  you  would  write  l>  in  front  of  the  ?ero. 


In  the  boxes  provided,  write  i.n  the  day  on  which  the  activity 
takes  place.  Please  use  the  abbreviations  THU,  FRI,  SAT,  SUN, 
MON,  TUE,  WED. 

Place  a  check  (/)  in  the  box  that  describes  the  activity 
involved  when  a  FACULTY-STUDENT  CONTACT  took  place.  For  the 
sake  of  consistency,  please  note  that 

--WARD  ROUNDS  include  case  presentations 

--ADMISSIONS  REVIEW  refers  to  the  review  of  admitting  histories 
•and  physicals  with  a  preceptor  or  resident 
--CLINICAL  ROUNDS  include  divisional  rounds,  departmental 
rounds,  and  visits  to  lab  and  x-ray 
--OPERATING  ROOM  includes  al 1  time  spent  in  the  Operating  Room 
and  Caseroom 

--NIGHT  CALL  includes  on  1 y  FACULTY-STUDENT  contact  time  during 
on-ca 1 1  period 

--CLINICS”,  SEMINAR,  and  LECTURE  are  self-explanatory  activity 
descriptions.  If  the  activity  involves  more  than  one  cate¬ 
gory,  please  check  only  the  one  which  describes  it  the  best 
in  your  judgement 

--OTHER  is  to  be  used  if  no  other  description  is  appropriate 


(Cont . ) 


' 

. 


. 


■ 


' 


■ 


Col.  (f;)  This  c  o  I  nit:’  i  it.  designed  to  estimate  the  p;  open  l  i  '>n  of  your 

activities  v/h  i  eh  a  r  c  relevant  to  cducat’onnl  purposes.  You 
file  requested  to  provide  the  estimate  according  to  your  best 
judgement,  rounded  to  the  nearest  10  pot  cent.  (The  inst  ;:e ro 
hos  been  filled  in.)  For  example,  if  the  activity  is  entirely 
for  Phase  III  students’  instruction,  100  per  cent  (of  tie)  is 
on  appropriate  estimate  and  you  should  write  10  in  front  of  the 
•  zero.  On  the  other  hand,  if  the  activity  has  a  dual  purpov  of 

patient  care,  and  demonstration  for  students,  you  may  estimate 
it  as  f>9  per  cent  (of  your  time),  or  any  fraction,  according  to 
your  judgement. 


Sampl  i  rig 

In  order  to  minimize  your  work,  sampling  methods  will  he  used 
instead  of  requesting  students  to  keep  a  record  for  the  entire  year. 
Consequently,  it  is  expected  that  one  selected  student-week  will  be  repre- ' 
senting  tots  s  tudent-weeks  on  the  average,  and  the  error  made  in  filling  cur 
this  form  will  be  multiplied  by  the  same  factor.  Therefore,  accurate  esti¬ 
mation  by  the  selected  student  is  an  essential  requirement  of  this  survey 
and  your  cooperation  will  be  greatly  appreciated. 


example 


The  attached  sheet  illustrates  a  hypothetical  T I  ME -SHEET  Tor  the 
week  starting  Thursday,  September  4,  1375.  The  hypothetical  student,  John 
Smith,  spent  k  hours  30  minutes  on  WARD  ROUNDS,  2  hours  50  minutes  in  CLINiC 
and  2  hours  in  OPERATING  ROOM  on  Thursday.  One  PRECEPTOR,  2  RESIDENTS, 

3  INTERNS,  and  k  Phase  III  students  were  present  at  the  WARD  ROUND,  while 
only  one  RESIDENT  and  one  Phase  III  student  (John  Smith  himself)  were  at  the 
CLINIC  activity.  John  Smith  estimates • that  the  educational  components  were 
80  per  cent,  100  per  cent,  and  60  per  can*;  of  the  time  for  the  WARD  ROUNDS , 
CLINICS,  and  OPERATING  ROOM  respectively. 
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Scaling  Techniques  and  Subjective  Evaluation: 

Two  Measures  Estimating  the  Instructional  Utility 

of  the 

Phase  III  Program  of  Medical  Education 


by 


Michael  Nusbaum 


Division  of  Health  Services  Administration 

HS A  531 

Dr.  C.B.  Hazlett 


May,  1976 
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I. 1  Background  to  the  Problem 

The  problem  of  evaluating  faculty  teaching  is  common  to 
all  universities.  A  widely-recognized  convention  of 
determining  professorial-  and  course-effectiveness  is 
through  the  use  of  course-evaluation  questionnaires,  by 
which  students  are  asked  to  comment  on  the  effectiveness  of 
the  major  component  parts  that  compose  an  instructor-course 
teaching  unit. 

However,  many  faculties  and  schools  [especially 
graduate  faculties]  do  not  follow  traditional  instructional 
formats,  which  usually  are  made  up  of  lectures,  laboratory 
sessions,  and  tutorials.  In  the  senior  year[s]  of  medical 
schools,  students  undertake  an  "internship"  program,  whereby 
they  spend  the  majority  of  their  time  in  the  hospitals 
receiving  on-the-job  training  for  the  various  specialties 
needed  to  produce  general  practitioners.  At  the  University 
of  Alberta  Faculty  of  Medicine,  students  in  Phase  III  [3rd 
and  4th  year]  are  required  to  take  a  fixed  number  of 
rotations  and  electives  through  selected  hospital 
departments.  Rotations  are  chosen  by  the  faculty 
administration,  and  electives  are  selected  by  the  students 

themselves. 


Within  any  rotation  or  elective,  a  student  participates 
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in  a  number  of  activities,  by  which  he  or  she  can  observe 
and  execute  the  various  tasks  expected  of  a  practicing 
physician.  Such  activities  may  include  ward  rounds,  clinical 
rounds,  admissions  review,  seminars,  lectures,  operating 
room  sessions,  and  night  calls. 

A  concern  of  the  faculty  administration  is  with  the 
effectiveness  of  the  various  activities  in  conveying  medical 
knowledge  to  the  students.  The  author  defines  effectiveness, 
in  the  context  of  this  paper,  as  the  amount  of  instructional 
benefit  transmitted,  by  each  of  the  activities,  to  the 
student.  Instructional  benefit  is  primarily  a  subjective 
measure,  but  may  include  (1)  the  proportion  of  allotted  time 
spent  by  a  student  receiving  direct  instruction,  (2)  an 
estimate  of  the  quality  of  instruction  generated  within  a 
particular  activity,  and  (3)  an  estimate  of  the  value  of  the 
activity  as  an  instructional  medium. 

At  the  present  time,  a  study  is  underway  primarily 
investigating  Phase  III  Faculty-Student  Contact  Time1.  One 


1  Portions  of  this  study  have  been  documented  as  part 
of  a  Masters  Thesis  [unfinished  at  present],  by  Nusbaum,  M.  , 
Division  of  Health  Services  Administration,  vacuity  of 
Graduate  Studies  and  Research,  University  of  Alberta, 
Edmonton,  Alberta. 
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of  the  issues  being  studied,  as  an  aside,  is  the  amount  of 
instruction  received  by  a  Phase  III  student  during  his/her 
participation  in  the  aforementioned  activities  of  rotations 
and  electives.  The  measure  is  reported  as  a  percentage  of 
time,  with  respect  to  the  amount  of  allotted  time,  for  each 
activity.  In  other  words,  a  student  is  asked  to  carry  a 
TIME-SHEET  around  to  the  various  activities  scheduled  for 
him/her,  for  a  period  of  one  week.  For  every  activity,  the 
student  subjectively  esimates  the  ”%  Instructional 
Component”,  and  jots  it  down  on  the  TIME-SHEET,  rounded  off 
to  the  nearest  10%. 

The  problem  with  this  type  of  measure  is  that  the 
student  is  assumed  to  have  the  ability  to  accurately  place  a 
value  on  the  amount  of  instructional  benefit  he/she  receives 
during  an  activity.  Practically  speaking,  this  is  very 
unlikely,  and  close  observation  of  the  Contact  Study  data 
reveals  a  somewhat  large  dispersion  of  responses  over 

similar  activities. 

The  author,  then,  has  chosen  to  apply  an  independent 
measure  of  instructional  benefit  to  the  Phase  III  students, 
to  examine  any  inconsistancies  in  the  data  collected  from 
the  Contact  Study.  The  independent  method  selected,  and 
discussed  in  this  paper,  is  Scaling. 
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I. 2  Statement  of  the  Problem 

As  previously  stated,  the  Contact  Study  presumed  a 
degree  of  expertise  on  behalf  of  the  respondents,  who  were 
asked  to  report  instructional  benefit  over  a  student- week2. 


As  for  the  independent  scaling  measure,  students  were 
asked  to  report  their  general  opinions  of  the  activities 
over  their  entire  experience  in  the  Phase  III  program. 


This  paper  will  attempt  to  answer  the  following 
questions: 


(1)  Is  it  relevent  to  ask  students  to  comment  on 

the  instructional  benefit  of  an  activity 
on  a  weekly  basis  as  compared  to  a 
general  opinion  over  all  Phase  III 
experience? 

(2)  Do  the  two  independent  measures  correlate 

significantly,  indicating  reliability  of 
response? 

(3)  What  are  the  potential  hazards  and  sources  of 

error  in  asking  the  students  to  respond 


2  a  stqident-week  is  defined  as  a 
consecutive  days  spent  by  a  Phase  III 
his/her  formal  rotation 


or  elective 


period  of  seven 
student  as  part  of 
schedule . 


4  *v  .  '  *J  ' 


1 

* 


144 


by  each  of  the  methods? 

(4)  What  inconsistancies  are  present  between  the 
two  methods,  and  can  any  inferences  be 
made  at  all? 

1-3  Definition  and  Classification  of  Scales 

Measurement  has  been  defined  as  "the  process  which 
permits  the  quantification  of  the  variables  so  that 
relationships  may  be  identified"  [Ware  and  Benson,  1975, 
p.576].  Three  stages  of  measurement  were  identified  by  Lord 
and  Novick  [1968]  as  (1)  selection  of  the  object  to  be 
measured,  (2)  identification  of  the  property  to  be  measured, 
and  (3)  determination  of  the  numerical  assignment  rule. 

Several  authors  have  attempted  to  categorize  "rules" 
according  to  their  mathematical  properties.  Torgerson  [1965] 
identified  characteristics  of  order,  distance  and  origin, 
and  developed  four  types  of  scales:  ordinal,  ordinal  with  a 
natural  origin,  interval,  and  ratio.  Stevens  [1968]  also 
identified  four  scale  types:  nominal,  ordinal,  interval,  and 
ratio. 


Nominal  scales  are  those  whereby  elements  to  be 
measured  are  merely  classified  qualitatively.  Ordinal  scales 
denote  rank  ordering  of  elements  with  respect  to  some 
property.  Interval  scales  "permit  statements  concerning  the 
equality  of  differences,  in  addition  to  statements  about 
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ordinal  position  and  equality"  [Ware  and  Benson,  1975, 
p.576].  Ratio  scales  differ  from  interval  scales  only  in 
that  they  relate  to  a  unique  natural  origin  [Torgerson, 
1965], 

Scales  based  on  distance  [interval  and  ratio]  require 
three  developmental  conditions  [Torgerson,  1965].  First,  the 
elements  must  be  able  to  be  ordered  in  a  "quasiserial" 
fashion.  Secondly,  deductions  must  be  made  concerning  the 
behavior  of  ratios  of  real  numbers.  Finally,  an  experimental 
test  must  be  made  to  determine  whether  the  distances  [  or 
ratios]  between  elements  behave  in  the  same  manner  as  the 
differences  [or  ratios]  between  real  numbers.  If  so,  the 
elements  may  be  assigned  real  numbers. 

I. 4  The  Law  of  Comparative  Judgement 

According  to  Torgerson  [1965],  Thurstone's  Judgement 
Scaling  Model  is  based  on  the  Law  of  Comparative  Judgement. 
The  model  states  that,  if  a  series  of  stimuli  are  given  to  a 
subject,  the  subject  can  respond  differentially  with  respect 
to  some  given  attribute.  The  researcher's  task  is  to  locate 
these  stimuli  on  a  psychological  continuum  [or  scale]  in 
such  a  way  as  to  account  for  the  responses  given  by  the 

sub jects. 


The  Law  of  Comparative  Judgement  is  merely  a  set  of 
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equations  relating  the  "proportion  of  times  a  stimulus  k  is 
judged  greater  on  a  given  attribute  than  any  other  stimulus 
j"  [  Torgerson,  1965,  p. 159].  The  equations  are  based  on  the 
following  assumptions: 


(1)  Each  stimulus,  when  presented  to  a 
subject,  gives  rise  to  a  "discriminal  process"3 
which  has  some  value  on  the  psychological 
continuum  of  interest. 


(2)  Due  to  momentary  fluctuations  of  a 
subject  in  his  reactions  to  stimuli,  a  given 
stimulus  may  excite  a  discriminal  process  with  a 
higher  or  lower  value  on  the  continuum  than  may 
expected.  It  is  postulated,  however,  that  the 
values  of  the  discriminal  processes  are  such  that 
the  frequency  distribution  is  normal  on  the 
continuum,  and  each  stimulus  has  a  normal 
distribution  of  discriminal  processes  associated 
with  it. 


(3)  The  mean  and  standard  deviation  of  the 
distribution  associated  with  a  stimulus  are  taken 
as  its  scale  value  and  discriminal  dispersion, 
respectively. 


(4)  If  two  stimuli  were  presented  together  to 
the  subject,  each  would  excite  a  discriminal 
process.  The  difference  in  discriminal  processes 
for  any  single  pair  of  stimuli  is  called  a 
discriminal  difference. 


The  distributions  of  discriminal  processes  can  be 


T 


3  A  discriminal  process  is  that  process  by  which  the 
subject  identifies,  distinguishes,  or  reacts  to  stimuli. 
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represented  graphically,  and  is  shown  in  Figure  1. 


The  complete  form  of  the  Law  of  Comparative  Judgement 
is  given  by  the  following  equation: 


r — - - — - - - - - - - 1 

(  I 

|  Sk-Sj  =  X  jk  {©  j  +  ©k  -  2R jk© j©k)  | 

I  I 

t _ _ _ ___ _ _ _ _ — - - -  j 


where: 

Sj,Sk  denote  the  scale  values  of  stimuli  j  and  k. 

©jr@k  denote  the  discriminal  dispersions  of  stimuli 

j  and  k. 

Rjk  is  the  correlation  between  the  pairs  of 

discriminal  processes. 

Xjk  is  the  normal  deviate  corresponding  to  the 

theoretical  proportion  of  times  stimulus  k  is 
judged  greater  than  stimulus  j. 


Figure  I:  Discriminal  Dispersions 
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1.5  Purpose  of  the  Study 

The  purpose  behind  this  study  was  to  construct  a  scale 
of  rotation  and  elective  activities,  such  that  the  scale 
values  rest  on  a  continuum  representing  instructional 
benefit. 


CONTINUUM 


< - 

less  instructionally 
beneficial 


- > 

more  instructionally 
beneficial 


I. 6  Experimental  Procedure 

Overton  and  Hazlett  [1976]  suggest  various  methods  of 
developing  a  scale,  including  ranking,  sorting,  single¬ 
stimulus  rating,  and  paired  comparisons.  The  Law  of 
Comparative  Judgement  requires  data  structured  in  such  a  way 
as  to  convey  ”the  proportion  of  times  stimulus  k  is  judged 
greater  than  any  other  stimulus  j”,  and  the  direct  method 
for  obtaining  these  proportions  is  by  paired  comparisons 
[Torgerson,  1965]. 


It  is  the  task  of  the  subject  to  indicate,  when 
presented  with  a  pair  of  stimuli,  which  of  the  pair  is 
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greater  with  respect  to  the  attribute  to  be  scaled.  The 
subject  must  select  only  one  of  the  stimuli  as  being 
greater,  and  ties  are  not  allowed. 

According  to  Overton  and  Hazlett  [1976],  the  method  of 
paired  comparisons  is  advantageous  because  (1)  it  forces  the 
subject  to  make  a  definitive  choice,  and  (2)  it  does  not 
force  transitivity  on  the  data,  in  that  the  subject  may  not 
desire  to  select  Stimulus  A  as  being  the  greater  of  the  pair 
for  all  pairs  in  which  it  appears. 

In  addition,  paired  comparison  selection  is  of  the 
lowest  order  in  terms  of  the  expected  expertise  required 
from  the  subject.  In  other  words,  the  task  of  choosing  one 
from  a  pair  is  much  clearer  to  understand  and  carry  out 
than,  say,  putting  a  value  on  stimulus. 

Paired  comparisons  is  disadvantageous  as  a  method  in 
that  a  subject  must  compare  every  stimulus  to  every  other 
stimulus,  and  this  becomes  a  likely  source  of  fatigue,  and 
hence  error. 

1. 7  Subjects  Included  in  the  Study 

The  subjects  chosen  to  be  included  in  this  study  were 
122  Phase  III  students  in  the  Faculty  of  Medicine  at  the 
University  of  Alberta  who  had  been  positive  respondents  to 
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— -  ------  — iSL—  Studx^ .  It  was  presumed  that  the  majority 

of  the  subjects  selected  had  partaken  in  each  of  the 
activities  at  least  once  during  their  Phase  III  experience. 

1.8  Stimuli 


The  stimuli,  or  objects,  presented  to  the  subjects  were 
the  7  most  frequently— attended  rotation/elective  activities: 
ward  rounds  (WR)  ,  admissions  review  (AR) ,  clinical  rounds 
(CR)  ,  operating  room  (OR)  ,  seminars  (S)  ,  lectures  (L)  ,  and 
night  calls  (NC). 

X.  9  Attribute 


The  attribute  of  interest  to  the  author  was  the  amount 
of  instructional  benefit  obtained  by  the  subjects  while 
attending  the  activities  in  question.  Instructional  benefit 
included  (1)  the  utilization  of  allotted  time  in  direct 
instruction,  (2)  the  quality  of  instruction  received  by  the 
subject,  and  (3)  the  utility  of  the  activity  as  a  mode  of 


4  The  subjects  comprised  a  100%  sample  of  those  who  had 
responded  to  the  Contact  Time  Study.  The  sampling  used  for 
the  Contact  Time  Study  was  a  two-stage,  stratified,  cluster 
sample,  and  details  on  this  sampling  plan  are  available  from 
the  author. 
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instruction.  Also,  the  attribute  related  to  the  subjects* 
general  opinion  over  his/her  entire  experience  as  a  Phase 
III  student. 

1. 10  Simplifying  Condition 

As  Torgerson  [1965]  suggests,  conditions  are  required 
to  further  simplify  Thurstone*s  model  in  order  to  obtain  a 
workable  set  of  equations.  This  study  utilized  simplifying 
Condition  B,  which  makes  the  assumption  that  correlation 
terms  [Rjk]  are  all  equal,  and  that  differences  between 
discriminal  dispersions  [0<+j-0<+k]  are  small. 

/ 

1.11  Research  Instrument 

The  research  instrument  took  the  form  of  a  one-page 
questionnaire,  distributed  with  a  cover  letter.  The 
questionnaire  was  divided  into  two  sections. 

The  first  section  asked  the  student  to  check  off  which 
of  'the  7  activities  he/she  had  ever  participated  in  during 
Phase  III. 


The  second  section  presented  the  subject  with  a  list  of 
paired  comparisons  for  the  7  activities.  This  required  the 
subject  to  compare  21  pairs  [ 7!/ (7-2) ! 2! ]  of  activities,  and 
indicate  for  each,  the  one  of  greatest  instructional 
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benefit.  The  order  in  which  the  pairs  appeared  on  the 
instrument  was  randomized  for  each  of  the  122  subjects, 
primarily  to  compensate  for  fatigue  error.  This  was 
accomplished  through  the  use  of  a  FCK/RTRAN  computer 
program,  which  generated  each  questionnaire  individually, 
and  with  complete  randomization. 

X.  12  Data  Collection 


The  research  instrument  was  distributed  by  mail  to  the 
122  subjects,  and  each  included  a  stamped,  return-addressed 
envelope.  The  number  of  returned  questionnaires  totalled 
103,  giving  a  response  rate  of  84.4%. 

The  returned  instruments  were  coded  and  keypunched  on 
computer  data  cards.  Through  the  use  of  SPSS5,  it  was  found 
that  66  of  the  103  subjects  [64%]  had  indeed  attended  all 
the  specified  activities,  17  [  17% ]  had  attended  6  of  the  7 
activities,  2  [2%]  attended  5  out  of  7,  and  the  remainder 
[  1 7% ]  either  attended  fewer  than  5  activities,  or  ommitted 
answering  one  or  more  of  the  comparisons. 


5  A  computerized  statistical  Package  for  the  Social 
Sciences,  McGraw-Hill,  1975- 
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1-13  Three  Basic  Matrices 

The  procedure  for  calculating  scale  values  utilized 
three  matrices:  Fr  Pr  and  Z.  Matrix  F  indicated  the  number 
of  times  a  stimulus  k  was  judged  greater  than  a  stimulus  j. 
In  other  words,  F12  represents  the  number  of  times  activity 
2  was  judged  more  instructionally  beneficial  than  activity 
1.  By  definition,  the  F-matrix  is  symmetrical,  in  that  F12  + 
F21  =  N  =  103. 


Matrix  P  was  constructed  directly  from  Matrix  F,  and 
represents  the  proportion  of  times  stimulus  k  was  judged 
greater  than  stimulus  j.  The  P-matrix  is  also  symmetrical, 
and  corresponding  elements  sum  to  unity. 

The  transformation  matrix.  Matrix  Z,  is  a  direct 
consequence  of  Matrix  P.  An  element  Zjk  is  the  unit  normal 
deviate  corresponding  to  the  element  Pjk,  and  is  obtained  by 
referring  to  a  table  of  areas  under  the  unit  normal 
distribution  curve. 

The  three  matrices  obtained  from  the  study  data  can  be 


found  in  Table  1. 
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TABLE  1 

Paired  Comparisons  of  Rotation/Elective  Activities 
by  Phase  III  Medical  Students  with  repect  to 

Instructional  Benefit 


Matrix  F:  The  number  of  times  each  activity  (k)  was 

judged  greater  by  Phase  III  students,  with 
respect  to  instructional  benefit,  than  other 
activities  (j) . 


r 

I 

L. 


k=  | 


HR 
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CR 


OR 


NC 
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TABLE  1  [continued] 

Matrix  P:  The  proportion  of  times  each  activity  (k)  was 

judged  greater  than  other  activities  ( j) . 
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1 
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TABLE  1  [continued] 

Matrix  Z:  Estimates  of  differences  between  scale  values 

for  each  pair  of  activites  (k-j) . 
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1.14  Scale  Values 


The  analytical  procedures  for  obtaining  scale  values 
from  the  Z-matrix,  under  simplifying  Condition  B,  are 
discussed  in  Torgerson  [1965,  pp. 179-185].  Table  2  shows  the 
resulting  scale  values  obtained  from  the  collected  data. 


- , - - - ) 

TABLE  2  | 

Scale  Values  for  Each  Rotation/Elective  Activity  | 


Rotation/Elective  Activities 


NC 

OR 

WR 

AR 

L 

CR 

s  I 

- r- 

1 

- j- 

1 

- r 

1 

i 

1 

f 

1 

1 

1 

1“ 

1 

1 

1 

Scale  1 
Values  | 

I 

1 

1 

1 

-0. 554| 

1 

i 

J 

-0.295| 

1 

1 

1 

1 

-0.  246| 

1 

1 

1 

1 

-0.101 | 

1 

1 

1 

1 

0.0781 

1 

1 

1 

0.347| 

1 

1 

0.770J 

J_ I - - « - 


I. 15  Goodness  of  Fit 

A  goodness  of  fit  test  was  performed  on  the  estimated 
scale  values.  The  purpose  of  the  test  was  to  determine 
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whether  or  not  the  attribute  in  question  [the  amount  of 
instructional  benefit]  could  be  utilized  in  determining 
scale  values,  given  actual  data.  In  other  words,  the  test 
was  used  to  demonstrate  whether  or  not  the  distances  between 
the  activities  on  the  scale  [according  to  the  attribute] 
behaved  in  the  same  manner  as  the  differences  between  the 
real  number  scale  values. 

According  to  Mosteller  [1951],  who  developed  a  goodness 
of  fit  test  for  scaling,  a  x  can  obtained  through  the 
use  of  an  inverse-sine  transformation  [see  Torgerson,  1965, 
pp.  185-7].  hosteller's  formula  yielded  a  x?  of  25.12198 
for  the  Phase  III  data.  Tested  against  the  null  hypothesis 
with  alpha=0*05  and  with  [  (n  (n-2) /2) -2n- 1 ]  degrees  of 
freedom  [.05X6  =12.6],  the  null  was  rejected,  indicating  that 

the  data  was  not  a  good  fit  to  the  theoretical  model. 

Because  the  author  was  uncertain  as  to  the  influence  of 
missing  data6  on  the  results  of  the  test,  the  scale  was 
recalculated  using  N=64  [all  subjects  had  participated  in 
all  the  activities,  with  no  missing  data].  Once  again,  the 


^  some  respondents  had  not  participated  in  all  the 
activities,  some  failed  to  select  only  one  o  e  or  ere 
pairs,  and  still  some 
ordered  pairs. 
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neglected  to  select  either  of  the 
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null  hypothesis  was  rejected  [  X  =  1  4.40543,  .05X6  =12-6]. 

The  test  results  indicated  that  the  scaling  technique 
utilized  in  this  study  was  not  appropriate  in  terms  of 
accurately  assessing  instructional  benefit  of  Phase  III 
activities.  However,  as  long  as  the  reader  is  aware  of  this 
inconsistancy,  the  author  will  assume,  for  the  remainder  of 
this  paper,  that  the  null  was  accepted,  and  that  the 
estimated  scale  values  did  indeed  fit  the  theoretical  model. 

I. 16  Phase  III  Faculty-Student  Contact  Time  Study 

For  comparative  [and  illustrative]  purposes,  the  "% 
Instructional  Components'’  of  the  FSCT  study  were  simply 
averaged,  to  form  a  scale,  for  each  of  the  contact 
activities  attended  by  the  students  from  September  4,  1975 

to  March  31,  1976.  These  averages  can  be  found  in  Table  3. 
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TABLE  3 

Average  %  Instructional  Component  for 
Each  Rotation/Elective  Activity 


Rotation/Elective  Activities 
NC  WR  OR  AR  CR  S  L 


- r- 

1 

/ 

— r- 

1 

T“ 

1 

r- 

1 

1 

Avg  %  | 

1 

1 

1 

1 

1 

1 

Inst.  | 

37 

1 

41 

1 

42 

1 

50 

(N)  | 

1 

(133) 

1 

1 

(813) 

1 

1 

(347) 

1 

1 

(215) 

_ _ _ i — 

1 

-  .1.-. 

1 

L- 

1 

-- L- 

l - , - j 


63  |  ,88 

(378)  1  (456) 


90 

(207) 


I 


Care  was  taken  so  that  all  of  the  respondents  to  the 
Scaling  instrument  were  also  included  in  the  FSCT  study 
results  reported.  A  bias  did  emerge,  however,  in  that  the 
respondents  sampled  in  the  Scaling  study  were  all  positive 
respondents  to  the  FSCT  study,  indicating  a  false¬ 
willingness  to  participate. 
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1-17  Spearman  Rho 

A  Spearman's  Rho  [ Rs  ]  was  calculated  with  respect  to 
the  two  scales  [ie.  from  the  Scaling  method  and  from  the 
Averaged  %  Instructional  Components].  Rs  is  defined  as  a 
non-parametric,  rank-order  correlation  coefficient, 
indicating  that  no  assumptions  are  made  regarding  the 
distribution  of  cases  on  the  variables  [such  as  there  are 
with  interval  and  ratio  scales].  Nie  [  1975,  p.  289  ]  denotes 
Rs  as  the  sum  of  the  squared  differences  in  the  paired  ranks 
for  two  variables  over  all  cases,  divided  by  the  sum  of  the 
squared  differences  in  ranks  [assuming  the  rankings  are 
independent  ]. 

The  correlation  was  performed  with  N=7  [activities], 
and  gave  a  value  of: 


r - - 

I  I 

1  Rs  =  0.85714  | 

I _ | 


1.18  Pearson  r 


The  Pearson  r  is  defined  as  the  product-moment 
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correlation  for  pairs  of  variables,  and  is  used  to  measure 
the  strength  of  their  relationship  in  terms  of  the 
parametric  properties,  Nie  [1975,  p.277]  refers  to  the 
Pearson  r  as  a  zero-order  correlation,  due  to  the  fact  that 
no  controls  for  the  influence  of  other  variables  are  made. 
Strength  of  relationship  indicates:  (1)  a  goodness  of  fit  of 
a  linear  regression  line  to  the  data,  and  (2)  the  proportion 
of  variation  [r2]  of  one  variable  explained  by  the  other. 


Again,  r  was  calculated  with  N=7,  and  the  result  was: 


i - 1 

I  I 

|  r  =  0.817073  [ 

[  I 

i - - 1 

i — - - i 

I  I 

|  r2  =  0,6676  | 

I  1 

i _ _ — - » 


1.19  fnmnxants  on  the  Two  Scales 

Due  to  the  parametric  nature  of  each  of  the  constructed 
scales  [in  that  they  both  are  characterized  by  distances 
between  scale  values,  and  not  merely  by  rank  order],  the 
Pearson  r  is  the  highest-order  correlation  coefficient 

therefore  the  most  appropriate  one  to  use. 


available,  and  is 
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When  a  one-tailed  T-test  was  applied  to  the  value  of  r 
with  5  degrees  of  freedom,  the  null  hypothesis  [Ho:  r=0]  was 
rejected  [Tcr=0.669],  A  two-tailed  test  also  yielded  a 
significant  result  [ Tcr=0. 754  ] ,  indicating  a  strong  positive 
correlation  between  the  two  scales.  However,  because  of  the 
low  value  for  N  [=7],  reliability  assumptions  could  only  be 
made  on  an  inferential  basis. 

The  question  of  whether  or  not  it  is  relevant  to  ask 
students  about  instructional  benefit  of  contact  activities 
on  a  weekly  basis  compared  with  a  basis  of  overall 
experience,  can  be  answered  affirmatively.  Because  the  two 
scales  correlated  significantly,  and  because  the  proportion 
of  variation  accounted  for  by  the  two  scales  [r2]  was  quite 
high  [67%],  it  is  safe  to  assume  that  the  two  independent 
measures  of  instructional  benefit  did  indeed  seek  the  same 
attribute,  and  responses  from  the  FSCT  study  are  more  or 
less  reliable. 

It  must  be  noted  that  the  goodness  of  fit  test 
performed  on  the  Scaling  results  indicated  that  the  model 
was  violated.  A  remedy  to  this  violation  may  rest  in 
changing  the  definition  of  "instructional"  benefit  to 
"educational"  benefit.  An  alteration  such  as  this  could 
possibly  strengthen  the  relationship  between  the  attribute 
and  the  stimuli.  However,  any  comparison  with  the  FSCT  study 
would  be  invalidated  by  the  change,  because  the  latter 
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strictly  reported  values  of  "instruction". 

I. 20  Summary 

In  conclusion,  given  the  constraints  of  the  two 
independent  measures,  it  can  be  stated  inf erentially  that 
there  is  a  significant  degree  of  reliability  between  the 
responses  of  the  two  studies,  and  that  Phase  III  students 
who  react  positively  to  the  instruments  are,  at  the  least, 
consistant  in  their  responses. 

Unfortunately,  the  Scaling  model  was  violated,  but  with 
some  definition  modification,  it  could  be  repeated  more 
successfully  at  some  future  point  in  time. 
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DIVISION 

alxh  .mzrvice:.'.  administration 

KLf.PHOMe  (AC>"»)  4  12-9^7  AMD  4**2-C409 


i  m c.  'j»'i i  v civb!  rv  o i *  r\ r > *i n  r a 
Lu.MorrroN.  ALf.sr.mA,  ca'nada 

TOG  2 G? 


31  August,  197^ 


Phase  Til  Students  (^Ith  Year) 

University  of  Alberta 
Faculty  of  Medicine 

Dear  Students: 

Re:  Phase  III  Faculty-Student  Contact  Time  Study 


Twelve  months  have  passed  since  you  were  first  imposed  upon  to  partici¬ 
pate  in  the  Phase  III  Faculty-Student  Contact  Time  Study.  During  that  time, 
817  TIME-SHEETS  were  mailed  out,  and  a  good  proportion  of  them  have  been  com¬ 
pleted  and  returned. 


At  the  present  time,  the  collected  data  is  being  analyzed,  and  some 
very  interesting  results  are  emerging.  Those  of  us  who  have  been  Involved 
the  study  wish  to  take  this  opportunity  to  express  our  gratitude  ioi  your 
part i ci pat  ion . 


ith 


In  order  to  evaluate  the  study  in  its  entirety,  we.  ask  you  to  complete 
just  one  more  task.  Attached  is  a  SHORT  guest i onna i 1 e  asi<ing  you  to  ct  iticize 
and  comment  on  the  content  oF  the  study  and  the  method  by  which  it  was  carried 
out.  Please  take  the  time  to  answer  all  the  questions  as  honestly  and  accurately 
as  possible.  When  you  have  finished,  fold,  staple  and  mail  tne  questionnaire 
back  to  us.  In  addition,  should  you  have  any  TIME-SHEETS  that  are  completed 
and  not  yet  submitted,  please  mail  these  In  as  well. 


Aga i n  ,  many  thanks , 


and  the  best  of  luck  in  all  your  future  endeavors.^ 

.  ,  .  ,  /f 

S  1  nccre 1 y , 


,y  "  -•  J  '/  .  [/"  / . .  £ 

'  £  A  'y  s  w<-< 

Ky  u  n  g^S .  Ba^,  P  h .  D  M  i  ch  ao  !  Nu  s  b  a  urn ,  B .  A . 

Associate  Professor  Study  Coord i ho  lor 
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f'ilASK  1X3,  FACULTY- STUDENT 
CONTACT  7  I  HE  STUDY 
FOL.I.OW-UP  QUESTIONNAIRE 


1.  Hove  you  ever  received  o  Contact  Study  TJME-SfiFET7 

□  VES 

fZ3  WO  O1  WO,  you  need  not  answer  any  more  questions 
the  questionnaire  and  accent  our  thanks). 


P 1  ease  ma i 1  in 


2.  How  many  Contact  Study  TIME-SHEETS  have  you  received  in  the  past  12 
months?  | — j — j 

3.  How  many  have  you  filled 'out  and  returned? 


Do  You  F ee  1  that  the  study  was  too  great  an  imposition  on  your  tirm-? 

P  YES 

□  NO 

5.  Did  the  study  investigators  make  it  clear  to  you  the  purpose  and  justi 
fi cat  ion  for  this  study? 

□  YES  .  ■ 

□  NO 


If  not,  suggest  how  this  may  be  improved  in  future  studies: 


6.  Did  you: 

f  |  carry  the  TIME-SHEETS  v/ith  you  during  the  week? 

Q  fill  out  the  TIME-SHEETS  at  the  end  of  the  week? 

O  Fill  out  the  TIME-SHEETS  later  than  at  the  end  of  the  week? 
]~]  not  fill  out  the  TIME-SHEETS? 

7,  Did  you  have  difficulty  in  understanding  the  instructions  as 
to  fill  out  the  TIME-SHEETS? 

□  Yrs 

□  NO 


to  how 


' 


x 
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2- 


8.  L-o  you  feel  that  the  1  I  Mi. -SHEETS  actually  reflected  your  student 
activity  as  far  as  direct  faculty  contact  time  is  concerned? 

□  YES 

□  NO 


9. 


n  tc  rn 


Bo  you  fed  that  the  TIME-SHEET  should  gather  any  additional  information 
not  already  being  collected? 

□  yes 

□  NO 

I  f  YE S ,  wha t  i n  forma t i on ? 


10.  Do  you  feel  that  the  mail  strike  last  November  seriously  affected 
whether  or  not  you  submitted  the* * TIME-SHEET? 

□  yes 

□  NO 

11.  Can  you  suggest  how  the  TIME-SHEET  might  be  redesigned  in  the  future 
to  improve  its  clarity  and/or  function? 


12.  Do  you  feel  that  estimates  of  Instructional  Component"  are  a 
realistic  measure  of  Faculty  contact? 

□  yes 

•  □  NO 

Comments:  _____  _  _ 


13. 


Have  you  filled  out  ALL  TIME-SHEETS  that  you  have  icceived? 
[  ]  YES,  i  have  done  so. 

\ '  \  MO,  I  have  not,  because: 

P  I  was  too  busy! 

□  I  forgot,  about  it. 

[  j  I  lost  the  ’ll  ME -SHEET, 
j  |  Other  (specify):  _ 


n 

1  was  not 

i  n teres  ted  . 

□ 

There  arc 

too  many  sui 

□ 

1  saw  no  ti 

se  for  this 

r« 
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Ml. 


The  Contact  Time  study  only  attained  about  a  $0?, 
general,  how  might  the  study  be  carried  out  at  a 
prove  the  rate  of  response? 


response  rate, 
future  date  to 


I  n 

i  m~ 


. 

Do  you  have  any  general  comments  and/or  criticisms? 

'  ■ 

PLEASE  FOLD,  STAPLE,  AMD  MAIL. 
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Physician  Interview  Format 
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UNIVERISTY  OF  ALHEP.TA 
FACULTY  0?  MEDICINE 

Phase  III.  Faculty -Student  Contact  Tima  Study 


M  D  N  a  m  e : 

Hospi  tal : 

#  students: 

1 .  D ef i n e  the  Act  iv it  ie s  i n 
an  average  Student-Week 


Date: 

Dept. : 

Teaching  Appt.: 
which  students  participate  in 


2*  State  the  goals  ana  objectives  of  your  particular 
program  in  terms  of  the  above  activities: 


3i  What  proportion  of  time  to  the  students  spend  at: 

Ward-  Rounds: 

Admissions  Review: 

Clinical  Rounds: 

Operating  Room: 

Clinics: 

Se mi  na  r: 

Lecture: 

Right  Call: 

Other  [Specify]: 

4  .  Vi  hat  st  a  f  £  (  re  so  urce  s  ]  are  a  v  a  ila  b  1  e 
interns? 

it  Preceptors: 

$  Interns: 

#  Residents: 


to  the  students 

Heeded: 
Needed : 
Needed : 


'  \ 


. 


' 
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A  n y  ccr. m c n  t s  a  bout  t he  Pha  se  III  P  rocj  ra in  ? 


Any  corn  merits  about  the  Study? 


6. 


■ 


